AN  EVALUATION  OF  INFORMATION  TECHNOLOGY 
(IT)  OUTSOURCING  DETERMINANTS  WITHIN  THE 
DEPARTMENT  OF  DEFENSE  (DoD) 

THESIS 

Alex  J.  Barelka,  Captain,  USAF 
AFIT/GIR/ENV/OlM-02 

DEPARTMENT  OF  THE  AIR  FORCE 
AIR  UNIVERSITY 

AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 

Wright-Patterson  Air  Force  Base,  Ohio 

APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 


20010612  U2 


The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the 
official  policy  or  position  of  the  United  States  Air  Force,  Department  of  Defense,  or  the 


U.  S.  Government. 


AFIT/GIR/ENV/0 1 M-02 


AN  EVALUATION  OF  INFORMATION  TECHNOLOGY  (IT)  OUTSOURCING 
DETERMINANTS  WITHIN  THE  DEPARTMENT  OF  DEFENSE  (DoD) 

THESIS 


Presented  to  the  Faculty 

Department  of  Systems  and  Engineering  Management 
Graduate  School  of  Engineering  and  Management 
Air  Force  Institute  of  Technology 
Air  University 

Air  Education  and  Training  Command 
In  Partial  Fulfillment  of  the  Requirements  for  the 
Degree  of  Master  of  Science  in  Information  Systems 


Alex  J.  Barelka,  B.S. 
Captain,  USAF 

March  2001 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 


AFIT/GIRyENV/OlM-02 


AN  EVALUATION  OF  INFORMATION  TECHNOLOGY  (IT)  OUTSOURCING 
DETERMINANTS  WITHIN  THE  DEPARTMENT  OF  DEFENSE  (DoD) 


Alexander  J.  Barelka,  B.S. 
Captain,  USAF 


Approved: 


Michael  G.  Morris  (Chairman) 


David  Petrillo  (Member) 


2^ 

date 

date 


Bradley  J.  Ayres  (Member) 


2^  foJ-O) 

date 


Acknowledgements 


First,  and  foremost,  I  would  like  to  thank  my  wife  for  having  the  patience  and  kindness 
for  putting  up  with  me  during  this  effort  and  making  sure  I  had  the  time  to  complete  the 
project.  I  could  not  imagine  having  done  it  without  her  support.  I  would  also  like  to  express 
my  sincere  appreciation  to  my  facility  advisor,  Maj.  Mike  Morris,  for  his  guidance  and  support 
throughout  the  course  of  this  thesis  effort.  His  help  and  periodic  sanity  checks  kept  me 
focused  and  on  track.  I  would  also  like  to  thank  the  men  and  women  of  the  National 
Reconnaissance  Office.  As  my  sponsor,  they  were  always  eager  to  provided  support  and 
assistance  to  me. 

Finally,  I  would  like  to  offer  my  sincere  thanks  to  the  experts  that  participated  in  my 
study.  All  were  senior  executives  in  the  Department  of  Defense  with  coimtless  other  priorities. 
However,  they  almost  all  found  the  time  to  complete  my  questionnaire  and  help  me  create  a 
richer  and  more  significant  product. 


Alex  J.  Barelka 


Table  of  Contents 


Page 


Acknowledgements . 

Table  of  Contents . ^ 

List  of  Figures . 

List  of  Tables . ^ 

Abstract . ^ 


L  Introduction . ^ 

General  Information . 1 

Definitions . ^ 

Outsourcing  and  Department  of  Defense . 3 

Research  Questions . 4 

Potential  Benefits . - . 6 

Thesis  Overview . ^ 

II.  Literature  Review . * 

Types  of  Outsourcing . ^ 

Interest  in  IT  Outsourcing . ^ 

Difference  Between  the  Private  and  Public  Sector . 10 

Government  Interest  in  IT  Outsourcing . 12 

Proposed  IT  Outsourcing  Theory . 13 

Financial  Determinants . 16 

Business  Determinants . 20 

Technical  Determinants . 27 

Political  Determinants . 31 

Security  Determinants . 35 

Environmental  Determinants . 41 

III.  Methodology . 45 

Introduction . 45 

Development  of  the  Conceptual  Model . 45 

The  Delphi  Technique . 46 


V 


Page 


Advantages  of  Delphi . 47 

Weaknesses  of  Delphi . 48 

Selection  of  Organziations . 51 

Selection  of  Experts . 53 

Development  of  the  Data  Gathering  Instrument . 55 

Round  One  Questionnaire . 56 

Results  and  Formulation  of  the  Second  Questionnaire . 56 

Roimd  Two  Questionnaire . 57 

Roxmd  Three  Questionnaire . 57 

Testing  of  the  Data  collection  Instrument . 58 

Faculty  Review . 58 

Questionnaire  Examination . 58 

Distribution  of  Data  Gathering  Instrument . 59 

Data  Processing . 59 

IV.  Data  Descriptions  and  Analysis . 60 

Introduction . 60 

Round  One . 60 

Results . 60 

Formulation  of  Second  Questionnaire . 66 

Round  Two . 69 

Results . 

Formulation  of  Third  Questionnaire . 74 

Round  Three . 79 

Results . 79 

Analysis  and  Findings . 82 

Summary . -87 

V.  Findings  and  Conclusions . - . 88 

Introduction . 88 

Macro  Perspective . 88 

Micro  Perspective . 93 

Financial . 93 

Business . 97 

Technical . 98 

Political . 100 

Security . 102 

Environmental . 103 

Additional  Factors . 105 

Limitation  and  Follow  on  Research . 105 


Suggestions  for  Future  Research 
Conclusions . 


Page 


Bibliography . 

Appendix  A:  Questionnaire  #1  and  Cover  Letter . 

Appendix  B:  Questionnaire  #2  and  Cover  Letter . 

Appendix  C;  Questionnaire  #3  and  Cover  Letter . 

Appendix  D:  Raw  data  from  round  #1 . 

Appendix  E:  Raw  data  from  round  #2..... . 

Appendix  F:  Raw  data  from  rormd  #3 . 

Appendix  G:  Inter-roimd  question  numbering  map . 

Appendix  H:  List  of  all  factors,  sub-factors,  and  associated  alphanumeric  designators 

Vita . 


108 

109 

111 

119 

131 

142 

148 

153 

157 

159 

165 

169 


vu 


List  of  Figures 


Figure  1 .  DoD  IT  Determinants  Outsourcing  Model . 

Figure  2.  Cost  Determinants . 

Figure  3.  Business  Determinants . 

Figme  4.  Technical  Determinants . 

Figure  5.  Political  Determinants . 

Figure  6.  Security  Determinants . 

Figure  7.  Environment  Determinants . 

Figure  8.  Group  Size  vs.  Error  Rate  (Dalkey,  1969:11) . 

Figure  9.  Reliability  vs.  Group  Size  (Dalkey,  1969:13) . 

Figxire  10.  Plot  of  Question  Means . 

Figure  1 1 .  Standard  Deviations  for  Rounds  1-2 . 

Figure  12.  Interquartile  Range  for  Roimds  1-2..... . 

Figure  13.  Standard  Deviations  for  Rounds  1-3 . 

Figme  14.  Interquartile  Range  for  Rovmds  1-3 . 

Figure  15.  Standard  Deviations  for  New  Items  between  Rounds  2-3 
Figure  16.  Interquartile  Range  for  New  Items  between  Rounds  2-3.. 


Page 

. 16 

. 17 

. 21 

. 28 

. 32 

. 38 

. 42 

. 52 

. 53 

. 65 

. 75 

. 76 

. 82 

. 84 

. 85 

. 85 


viii 


List  of  Tables 


Page 

Table  1.  IT  Outsourcing  Frameworks . 13 

Table  2.  Study  Participants . 34 

Table  3.  Example  of  Raw  Data . 61 

Table  4.  Example  of  Data  Sorted  Based  on  Item  Type . 62 

Table  5.  First  Round  Data  Sorted  Based  on  Mean . 63 

Table  6.  Sub-set  of  Questions  Selected  for  Second  Survey . 67 

Table  7.  New  Questions . 69 

Table  8.  Original  Second  Round  Data  Sorted  Based  on  Mean . ..71 

Table  9.  Re-worded  Second  Round  Data  Sorted  Based  on  Mean . 73 

Table  10.  Example  of  Stability  Measurement  Calculation  in  Rovmd  Two . 77 

Table  1 1 .  Stability  Measurements  for  Questions  Retained  for  Survey  Three . 78 

Table  1 2.  Third  Round  Data  Sorted  Based  on  Mean . 81 

Table  13.  Stability  Measurements  for  Survey  Three  Questions . 83 

Table  14.  Stability  of  New  Questions . 86 

Table  15.  Top  Ten  Highest  Rated  Items . 87 

Table  16.  Number  of  Questions  Retained  from  Rovmd  One  to  Three . 89 

Table  17.  Number  of  Questions . 89 

Table  18.  Number  of  Questions  Retained  from  Rovmd  One  to  Two  and  Two  to  Three . 92 

Table  19.  Ranking  of  Final  Items  in  Each  Rovmd . 95 

Table  20.  Percentage  of  Questions  with  Bimodal  Distributions . 1 07 

ix 


d 


AFIT/GIR/ENV/OlM-02 


Abstract 


Each  year  the  Department  of  Defense  (DoD)  spends  billions  of  dollars  on  information 
technology  (IT)  outsourcing.  However,  little  formal  or  academic  guidance  has  appeared  that 
oqjlains  how,  why,  or  even  when  this  occurs.  This  study  presents  and  evaluates  several 
determinants  that  may  impact  a  decision  to  outsource  information  technology  systems  in  the 
Department  of  Defense. 

It  begins  with  the  development  of  a  conceptual  model,  which  was  created  using  semi- 
structured  interviews  and  an  extensive  literature  search.  This  model  was  then  matured  into  an 
analytic  version  by  using  the  Delphi  method,  which  is  an  accepted  methodology  to  use  when 
insuflScient  or  no  appHcable  data  exists,  the  required  data  is  too  expensive  to  obtain  and 
analyze,  or  the  problem  variables  and  their  interaction  are  not  clearly  known. 

The  results  seem  to  suggest  that  while  some  determinants  are  more  important  than 
others,  the  decision  to  outsource  IT  in  the  DoD  is  a  multifaceted  one.  This  is  consistent  with 
similar  research  done  in  the  private  sector.  The  results  also  seem  to  suggest  that  in  the  area  of 
IT  outsourcing,  the  DoD  seems  to  be  experiencing  much  the  same  evolution  the  private  sector 
did  and  that  each  organization  has  a  slightly  different  focus  and  requirement  set. 
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AN  EVALUATION  OF  INFORMATION  TECHNOLOGY  aX)  OUTSOURCING 

DETERMINANTS  WITHIN  THE  DEPARTMENT  OF  DEFENSE  (DoD) 

I.  Introduction 

General  Information 

In  the  year  2000  total  military  spending  will  top  approximately  $289.8B  (in 
constant  fiscal  year  2001  dollars)  and  total  military  active  duty  manning  will  hover  around 
1.383M.  This  is  down  40%  and  36.2%  respectively  since  1986  (Budgets,  2000;  People, 
2000;  USAF,  1990).  Over  the  same  14-year  period,  our  society  has  seen  an  explosion  in 
the  number,  type  and  capability  of  telecommunication  and  information  systems.  As  one  of 
the  most  pronovmced  examples,  the  Internet  has  grown  from  a  small  research  project  into 
one  of  the  largest  revolutions  in  information  sharing  and  commerce.  It  is  therefore  not 
surprising  that  the  importance  of  information  and  information  technology  in  the  military  has 
also  grown.  Retired  General  and  former  Air  Force  Chief  of  Staff,  Ronald  R.  Fogleman  has 
characterized  the  increased  inportance  of  information  as  having  an  “ascending  and 
transcending  influence  on  military  forces”  (Fogleman,  1995).  As  such,  he  has  labeled  it  the 
fifth  dimension  of  warfare,  following  land,  sea,  air,  and  space  (Fogleman,  1995).  In 
addition.  Joint  Vision  2010,  the  conceptual  template  for  future  US  warfighting,  states  “the 
emerging  importance  for  information  superiority  will  dramatically  impact  how  well  our 
Armed  Forces  can  perform  it  duties  in  2010”  (CJCS,  1996:8). 

However,  ensuring  information  superiority  may  not  simply  be  a  matter  of 
developing  and  fielding  superior  communications  and  computer  equipment.  Former 
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Deputy  Secretary  of  Defense  Dr.  John  Harare  states  that,  “security  in  the  virtual  world  is  as 
much  a  process  of  management  approach  and  attention  as  it  is  of  technology”  (Harare, 
1998).  Therefore,  it’s  important  that  leaders  not  only  concentrate  on  the  technology  that 
drives  these  systems,  but  also  on  the  management  that  controls  them.  Of  course,  this  must 
all  be  done  in  an  environment  where  military  outlays  and  personnel  levels  continue  to 
shrink. 

One  strategy  the  Department  of  Defense  is  currently  investigating  as  a  means  to 
provide  the  required  level  of  information  services  to  its  members  while  decreasing  the 
overall  cost  of  information  systems  is  outsourcing. 

Definitions 

There  are  numerous  perspectives  for,  and  definitions  of  outsourcing.  As  an 
example,  it  has  been  considered  using  labor  market  economics  (Slaughter,  1996), 
management  perspectives  (Keene,  1 998b)  resource  dependency  perspectives  (Pfeffer, 
1978),  and  agency  theory  (Hancox,  1999)  just  to  name  a  few.  This  multifaceted  nature  of 
outsourcing  has  lead  to  the  development  of  several  different  and  sometimes  conflicting 
definitions.  In  fact,  a  1991  America  survey  of  chief  information  officers  concluded,  “There 
is  little  precision  in  the  term  outsourcing"  Much  of  this  ambiguity  can  be  attributed  to  the 
constant  change  in  the  scope  of  IT  outsourcing.  In  the  early  1960’s,  organizations  would 
hire  outside  firms  to  simply  batch  process  large  amounts  of  data,  or  manage  small 
inejqjensive  systems.  As  time  progressed,  these  arrangements  grew  into  the  mega-deals, 
such  as  the  Kodak/IBM  arrangement  that  have  become  familiar  today  (Lacity,  1993). 
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However,  despite  the  differences  and  contradictions  in  the  definition,  there  does 
appear  to  be  an  agreed  upon  core  in  almost  everyone’s  version.  This  core  will  he  the 
working  definition  used  for  this  research: 

“Information  technology  (IT)  outsourcing  is  the  third  party  provision  of  IT 
products  and  services”  (Hancox,  1999). 

A  third  party  can  be  considered  the  agent  in  a  principle-agent  relationship  (Carlton,  2000). 
The  reference  to  services  includes  management  and  control  fimctions.  If  these  services  are 
excluded,  the  relationship  tends  to  he  more  of  a  traditional  contracting  out  arrangement 
(Keene,  1998b). 

Outsourcing  and  Department  of  Defense 

Almost  all  firms  outsource  to  some  degree.  For  many  years  businesses  have  been 
outsourcing  their  systems  development  efforts  to  software  houses.  (Earl,  1996)  Even 
military  organizations  traditionally  considered  in-house  work  centers,  such  as  the  Air  Force 
Research  Laboratory  and  other  military  run  research  and  developmental  centers,  perform  a 
certain  degree  of  outsourcing.  In  today’s  environment  of  reduced  military  personnel  and 
contractor  partnerships,  it  is  difficult  to  conceive  of  a  situation  where  a  single  Department 
of  Defense  (DoD)  organization  would  have  a  sufficient  number  of  Government  personnel 
with  the  correct  skill  set  and  experience  to  design  and  build  the  advanced  intelligence  and 
weapon  systems  required  by  today’s  military.  At  a  minimum,  the  Government  would  need 
to  hire  civilian  experts  and  consultants  when,  in  the  planning  stages,  their  skill  set  was 
incorrectly  not  thought  to  be  significant. 
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Due  to  lower  DoD  budgets  and  subsequent  manpower  drawdowns,  the  DoD 
currently  performs  even  less  in-house  developmental  work  than  it  once  did  (Brower,  1996). 
No  longer  are  external  vendors  used  to  simply  execute  large-scale  fabrication  efforts 
(Bryce,  1996).  Rather,  they  are  now  being  used  for  research  and  development  as  well 
transitional  management  activities.  Therefore,  any  remaining  questions  regarding  whether 
outsourcing,  or  its  derivative  arrangements,  should  be  used  usually  revolves  around  the 
operations  and  maintenance  (O&M)  services  it  offers.  Consequently,  this  study  will  focus 
on  this  area. 

Research  Questions 

This  study  is  an  inductive  examination  of  a  proposed  theory  regarding  information 
technology  determinates  in  the  Department  of  Defense.  It  will  be  conducted  using  the 
Delphi  Method  which  is  “a  method  to  systematically  collect,  evaluate,  and  tabulate 
independent  opinion  without  group  discussion.”  (Tersine,  1976:51).  The  Delphi  experts 
used  to  support  the  study  were  primarily  Chief,  or  deputy  Chief,  Information  Officers 
chosen  from  Department  of  Defense  or  intelligence  community  agencies.  The  final  output 
of  this  study  wiU  be  an  evaluation  of  the  proposed  theory.  The  research  focuses  on 
outsourcing  arrangements  that  have  a  direct  impact  on  the  entire  orgaimation,  not  just  a 
specific  portion  of  it.  Therefore,  only  monetarily  large  outsourcing  arrangements  will  be 
considered.  Typically  these  deals  run  in  excess  of  one  million  dollars.  It  shoiild  be 
emphasized  that  this  research  focuses  on  the  determinants  that  affect  a  decision  to 
outsource,  not  the  determinants  that  affect  the  amount  of  outsourcing.  A  decision  to 
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outsource  will  precede  any  decision  to  determine  the  level  of  outsourcing  to  be  used. 
Consequently,  the  primary  research  question  (dependant  variable)  being  addressed  is: 

1.  What  determinants  affect  an  outsourcing  decision  in  the  Department  of 

Defense,  Air  Force  or  IntelUgence  Community  agencies. 

A  sub-question  is; 

la.  Whether  the  proposed  determinants  have  a  positive  or  negative 

relationship  on  the  dependant  variable. 

This  study  does  not  focus  on  any  one  particular  model,  but  rather  considers  all  pre¬ 
event  IT  outsourcing  determinants.  It  draws  upon  a  number  of  existing  theoretical  models 
from  the  outsourcing  literature  that  attempt  to  describe  the  IT  outsourcing  decision 
process.  Presently  however,  none  of  these  models  focus  on  the  DoD.  This  is  a  difiBcult 
topic  to  capture  since  these  issues  are  “often  murky,  hidden  behind  euphemisms,  perceived 
differently  by  different  stakeholder  groups,  and  generally  not  easily  analyzed”  (Hirschheim, 
2000:100).  However,  without  this  information  it  is  difficult  to  make  any  predictions  about 
the  outcome  of  future  outsourcing  decisions,  except  in  the  most  general  case  of  lessons 
learned.  It  is  hoped  that  by  providing  a  comprehensive  framework  for  why  these  decision 
are  &st  made,  future  decisions  can  be  analyzed  using  it,  which  will  hopefully  lead  to  a 
better  forecast  of  the  outcome. 

This  is  similar  to  the  strategy  behind  any  information  systems  planning  (ISP) 
process.  ISP  is  “an  orderly  means  of  assessing  the  information  needs  of  an  organization 
and  defining  the  systems,  databases,  and  technologies  that  will  best  satisfy  those  needs” 
(Hoffer,  1999:168). 
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The  first  step  in  ISP  is  to  describe  the  current  situation.  If  you  don’t  know  where 
you  are  it  is  difficult  to  get  to  where  you  want  to  be.  The  crux  of  this  research  will  be  to 
develop  a  theory  that  explains  this  for  Department  of  Defense  IT  outsourcing.  The  next 
ISP  step  is  to  describe  the  target  situation,  trends  and  constraints.  This  will  usually  lead  to 
a  better  understanding  of  a  firm’s  desired  IT  outsourcing  end-state  and  therefore,  will  help 
determine  what  requirements  are  needed  to  satisfy  using  an  outsomcing  strategy.  The  last 
step  is  to  develop  a  transition  strategy  and  plan.  Once  you  know  where  you  are  and  where 
you  want  to  go,  formulating  of  a  path  to  get  fi'om  one  to  the  other  should  be 
straightforward.  The  transition  plan  should  be  a  detailed  as  possible  and  provide  sufficient 
guidance  to  aU  “levels  of  management  concerning  what  needs  doing,  how,  when  and  by 
whom  in  the  organization”  (Hoffer,  1999:  176).  It  is  hoped  that  flow-up  research  in  this 
area  will  lead  to  the  development  of  the  last  two  items. 

Potential  Benefits 

By  creating  a  theory  for  the  IT  outsourcing  determinants  in  the  military,  it  is 
hoped  that  a  better  understanding  will  be  gained  of  what  direction  this  method  of  IT 
governance  should  head  and  how  the  goals  created  for  it  can  be  attained. 

Additionally,  the  theory  could  potentially  have  a  more  immediate  benefit.  Since  the 
final  result  will  essentially  be  a  list  of  validated  determinants,  benefits,  and  risks  associated 
with  IT  outsourcing,  it  could  be  used  to  guide  future  IT  outsourcing  decision  makers. 

Also,  with  slight  modifications,  it  might  be  able  to  serve  as  the  basic  evaluation  criteria  for 
an  IT  outsourcing  source  selection  conunittee. 
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Thesis  Overview 


The  next  chapter  will  detail  the  development  of  a  conceptual  model.  This  model 
was  created  from  the  literature  and  interviews.  Chapter  3  will  describe  how  this  research 
was  conducted  and  focuses  on  the  Delphi  process  used  to  verify  the  conceptual  model. 
Chapter  4  will  provide  the  results  from  the  Delphi  experiment.  Finally,  chapter  5  will 
provide  any  insight  into  why  the  results  showed  what  they  did,  list  limitations  of  the 
research  and  provide  suggestions  on  follow-up  activities. 
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IL  Literature  Review 


Types  of  Outsourcing 

As  briefly  discussed  in  Chapter  1,  outsourcing  can  be  viewed  from  a  variety  of 
perspectives.  From  the  point  of  view  of  labor  market  economics,  outsourcing  can  be  a 
firm’s  response  to  the  costs  and  disadvantages  associated  with  the  traditional  permanent 
work  arrangement  that  arise  from  changes  in  technology  and  the  environment  (Slaughter, 

1 996).  Such  a  response  provides  flexibility  to  the  firm  since  they  do  not  have  to  make  a 
full  and  permanent  investment  to  obtain  additional  personnel  to  solve  what  might  be  a 
temporary  problem. 

According  to  the  General  Accoxmting  Office  (GAO)  outsourcing  is: 

When  a  government  entity  remains  fully  responsible  for  the  provision 
of  affected  services  and  maintains  control  over  management 
decisions,  while  another  entity  operates  the  functions  or  performs  the 
service.  This  approach  includes  contracting  out,  the  granting  of 
franchises  to  private  firms,  and  the  use  of  volimteers  to  deliver  public 
services.  (GAO,  1997) 

In  this  definition,  the  outsourcing  customer  retains  management  functions  and  control 
while  the  provider  simply  supplements  the  performance  of  the  firm.  However,  some  argue 
that  this  is  not  a  complete  definition.  They  contend  that  assets  and  staff  members  need  to 
be  transferred  from  the  principal  to  the  agent  before  outsourcing  can  occur  (Willcocks, 
1993).  Others  are  not  so  radical  in  their  approach,  but  still  propose  that  outsourcing 
includes  the  partial  or  frill  transfer  of  management  to  the  vendor  (Keene,  1998b). 

Recently,  the  terms  Smart  sourcing,  selective  sourcing  and  right  sourcing  have 
been  introduced  to  describe  the  selective  use  of  outsourcing  as  opposed  to  a  complete 
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transfer  of  the  work  to  a  vendor.  The  optimism  attached  to  this  idea  is  a  hope  from  firms 
that  outsourcing  works  best  when  only  selected  systems  are  outsourced  (Lacity,  1996). 

StHl  other  firms  have  investigated  cooperative  outsourcing.  In  this  arrangement  the 
vendors  profits  are  based  on  agreed  to  improvements  in  the  customer’s  basic  corporate 
performance,  such  as  earnings,  performance  against  budget,  and  competitive  position 
(Scott,  1995). 

Additionally,  some  firms  that  choose  to  outsource  their  IT  are  now  even  re- 
intemalizing  these  services  under  the  heading  of  insourcing  (Meyer,  1994).  They  feel  that 
the  e3q)ected  benefits  of  outsourcing  never  really  materialized  and  that  having  more  control 
over  their  IT  assets  is  better  than  having  less. 

It  should  also  be  noted  that  outsourcing  and  privatization  are  usually  considered 
different  arrangements.  Typically,  privatization  is  used  as  a  catch-all  phrase  used  to 
describe  any  new  use  of  the  private  sector  such  as  the  divestiture  of  government  facilities 
(Tighe,  1997).  Therefore,  outsoincing  can  be  considered  a  type  of  privatization. 

Interest  in  IT  Outsourcing 

Recently,  information  technology  (IT)  outsourcing  has  been  riding  a  wave  of 
popularity  in  the  private  sector  since  many  companies  are  now  considering  IT  as  a  utility  or 
commodity  and  therefore  able  to  be  performed  by  any  number  of  firms.  (Lacity,  1993). 
Consequently,  IT  outsourcing  has  became  a  serious  strategic  choice  for  firms  (Loh,  1992). 
Lower  costs  and  higher  performance  have  become  mantras  of  this  movement. 

However,  firms  are  learning  that  IT  is  different  from  other  business  functions 
because  it  permeates  an  entire  organization.  It  touches  nearly  every  aspect  of  a  business 
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and  therefore  cannot  be  treated  as  a  discrete  entity  like,  security,  logistics  or  advertising. 
Therefore,  they  are  discovering  that  IT  outsourcing  presents  different  legal  and  commercial 
issues  when  compared  to  more  traditional  outsourcing  arrangements.  It  s  been  argued  that 
the  complexity  and  changing  nature  of  IT,  as  well  as  the  legal  framework  governing  its  uses 
typically  cause  these  differences  (Antonucci,  1998). 

Due  to  these  different  qualities,  numerous  methods  have  been  applied  to  describe 
IT  outsourcing.  Some  research  has  used  popular  conceptual  models  such  as  principal 
agent  theory,  transaction  cost  economics,  core  competencies,  agency  theory,  partnerships 
and  organizational  fectors  (Brown,  1999;  Kim  1998,  Hancox,  1999).  Some  have  described 
IT  outsourcing  agreements  as  being  either  facilities  management,  applications  management 
or  managed  networks  (Rebeiro,  1996).  Others  have  attempted  to  capture  differences  in  IT 
outsourcing  by  creating  categories  for  each  such  as  network  services,  service  retention, 
service  transfer,  and  asset  transfer  (Takac,  1994). 

Difference  Between  the  Private  and  Public  Sector 

While  many  studies  exists  that  examine  IT  outsourcing  in  the  private  sector,  almost 
none  consider  its  application  to  the  pubic  realm,  especially  the  Department  of  Defense 
(McTeman,  1997).  However,  significant  and  important  differences  can  potentially  exist 
between  private  and  public  sector  firms  with  regards  to  IT  outsourcing. 

Many  of  these  differences  might  exist  due  to  the  presence  of  inherently 
governmental  factors  that  private  firms  do  not  need  to  consider,  such  as  the  Information 
Technology  Management  Reform  Act  (ITRMA)  of  1996,  statues,  federal  acquisition 
regulations  and  local  requirements  and  guidelines  issued  by  each  agency  (Keene,  1998a; 
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Keene,  1998b).  The  Balanced  Budget  Act  and  the  Office  of  Management  and  Budget 
(0MB)  scoring  are  just  a  couple  of  specific  factors  that  do  not  need  to  be  considered  when 
making  an  IT  outsourcing  decision  in  the  private  arena  (Keene,  1998a). 

Studies  in  the  United  Kingdom  (UK)  have  also  shown  fundamental  differences  exist 
between  public  and  private  sector  firms  vdth  regards  to  IT  outsourcing  (Hancox,  1999). 
Not  surprisingly,  some  of  the  largest  reported  differences  are  in  the  political  arena.  It 
needs  to  be  remembered  that  a  public  sector  organization  is  not  merely  a  provider  of  goods 
and  services,  it  is  also  a  governmental  and  political  institution  constituted  by  local  election 
(Hancox,  1999).  Also,  private  sector  IT  outsourcing  decisions  are  usually  based  on 
profitability  and  financial  effects  (Smith,  1998;  Loh  1992).  However,  public  sector 
organizations  sometimes  base  similar  decisions  on  more  personally  motivated  criteria 
(Hancox,  1999).  Other  differences  between  public  and  private  sector  organizations  that 
affect  IT  outsourcing  include;  stricter  procedures,  more  required  coordination,  political 
difficulties,  and  policy  shifts  due  to  changing  administrations  (Hancox,  1999). 

The  type  and  purpose  of  IT  systems  in  the  government  also  seems  to  be  inherently 
different  than  those  in  the  private  sector.  DoD  uses  of  IT  are  typically  very  focused,  one- 
of-a-kind  applications,  such  as  space  exploration,  advanced  research  and  development,  or 
battlefield  management.  However,  private  sector  firms  use  IT  for  broad  ranging 
applications  such  as  business  support  or  administrative  applications  (Jones,  1999). 

Consequently,  it  is  clear  that  Government  IT  outsourcing  has  the  potential  to  be 
inherently  different  fi-om  similar  arrangements  in  the  private  sector.  Therefore,  some  argue 
that  a  different  perspective  for  IT  outsourcing  is  required  for  the  Government  (Keene, 
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1 998b).  Gaining  this  perspective  is  even  more  important  when  it  is  realized  that 
government  agencies  outsource  over  twice  as  many  IT  services  as  private  firms  (Jones, 

1999). 

Government  Interest  in  IT  Outsourcing 

Currently,  there  is  concern  fi-om  Congress  that  Government  Agencies  are  not 
responding  to  the  growth  of  IT  in  an  appropriate  manner.  Representative  Sanford  Bishop 
of  Georgia  has  stated  in  congressional  session  that,  “The  telecommumcations  and 
information  technology  industry  appears  as  a  whirlwind  with  the  NSA  [National  Security 
Agency],  at  the  moment,  trailing  in  its  wake”  (US  House,  2000).  Comments  like  this  lead 
to  the  creation  of  Project  Groundbreaker,  a  comprehensive  study  to  outsource  almost  all  of 
the  NSA’s  non-core  IT  activities.  While  the  potential  fimding  associated  with  this  project 
is  unknown,  it  has  been  estimated  at  over  $5B  spread  over  10  years  (Verton,  2000).  Given 
these  figures  and  the  high  level  attention  being  paid  to  IT  and  IT  outsourcing,  it  is  not 
surprising  that  the  Department  of  Defense  has  increased  its  interest  in  outsourcing  as  a 
mode  of  IT  governance. 

0MB  Circular  A-76,  Performance  of  Commercial  Activities,  directs  most  all  of  the 
DoD’s  outsourcing  efforts.  The  motivation  behind  this  process  is  to  emphasize  that  in  the 
process  of  governing,  the  Government  should  not  compete  with  its  citizens”  (0MB,  1983). 
Therefore,  the  conclusion  is  made  that  the  Government  relies  on  “commercial  sources  to 
supply  products  and  services  the  Government  needs”  (0MB,  1983).  The  supplement  goes 
even  further  to  state  that  an  outside  source  can  be  selected  even  it  is  more  expensive,  so 
long  as  it  provides  the  best  value  to  the  government  (0MB,  1996).  Armed  with  these  two 
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ideas,  the  DoD  can  seek  outside  vendors  for  its  IT  systems  and  work  towards  not  only 
reducing  costs  but  increasing  value  as  well. 


Proposed  IT  Outsourcing  Theory 

When  considering  IT  determinants  it  is  helpful  to  classify  them  into  categories. 
Several  frameworks  fi)r  this  purpose  have  been  developed.  Some  examples  of  these  are 
listed  in  Table  1  below. 


Table  1.  IT  Outsourcing  Frameworks 


Model 

Categories  used  in  model 

Loh,  1992 

Business  competence 
Business  governance 

IT  competence 

Smith,  1998 

Cost  reduction 

Focus  on  core  competence 

IS  capability 

Security  and 
Environmental 

Gurbaxani,  1996 

IS  improvement 

Business  impact 
Commercial  exploitation 

Lacity,  1994 

Financial 

Business 

Technical 

Political 

Flexibility  seems  to  be  an  underlying  goal  for  each  of  these  categories.  Today’s 
business  environment  can  change  so  quickly  it  is  important  to  weave  as  much  flexibility 
into  a  corporate  strategy  as  possible.  Therefore,  IT  systems  must  also  be  able  to  respond 
quickly  to  changing  demands  (Antonucci,  1998).  Those  that  outsource  hope  it  will  lead  to 
increased  flexibility,  yet  no  single  category  can  exclusively  claim  this  as  a  goal.  For 
example,  since  IT  evolves  so  quickly,  by  the  time  a  firm  purchases  hardware  and  trains  its 
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staff  on  how  to  use  it,  the  technology  may  be  obsolete  (Rebeiro,  1996;  Slaughter,  1996). 

In  this  case  both  business  and  technology  categories  are  impacted.  It  is  also  hoped  by 
some  that  IT  outsourcing  can  enhance  financial  flexibility,  by  providing  such  services  as 
guaranteed  pricing  (Sweeny,  1996).  Consequently,  it  can  be  seen  that  flexibility  is  a  goal  of 
all  IT  outsourcing  determinants. 

It  has  been  suggested  that  because  of  this  flexibility,  outsourcing  is  even  more 
important  in  the  public  sector  since  such  organizations  are  extremely  cautions  about 
expanding  their  human  resource  base  due  to  work  rules  and  implicit  employment 
guarantees  (Slaughter,  1996).  Therefore,  these  firms  might  use  outsourcing  to  temporarily 
hire  a  contractor  for  the  same  task. 

Each  of  the  fi-ameworks  presented  in  Table  1  has  strengths  and  weakness  to  base  a 
theory  of  military  IT  outsourcing  determinants  on.  One  notable  weakness  is  the  inclusion 
of  determinants  intrinsic  to  the  private  sector,  such  as  cash  needs  and  financial  leverage 
(Loh,  1992).  However,  that  is  not  to  say  that  fiscal  elements  should  be  discounted  when 
considering  military  determinants.  Cost  related  factors  could  quite  easily  be  the  largest 
determinant  for  military  IT  outsourcing.  As  an  example,  during  the  development  of  the 
conceptual  model,  one  interviewee  commented  that  outsourcing  provides  budget  stability 
(Barclay,  2000).  He  ejqjlained  that  in-house  re-capitalization  expenditures  for  IT 
sometimes  make  an  easy  target  during  budget  reductions.  However,  if  such  a  function 
were  provided  for  in  a  contraict,  the  funding  would  be  more  difficult  to  raid  due  to 
contracting  regulations. 
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In  addition,  when  compiling  a  list  of  IT  outsourcing  determinants,  it  is  important  to 
not  only  consider  the  benefits  associated  with  the  arrangement,  but  any  potential  risks  as 
well.  Few  decisions  will  be  made  without  considering  the  possible  repercussions. 
Consequently,  it  is  helpful  to  review  any  hterature  that  pertains  to  IT  outsourcing  risks 
when  attempting  to  compile  a  list  of  determinants.  Some  have  categorized  these  risks  as 
being  either  pohtical,  managerial/technical,  legal,  organizational,  or  economic  (Keene, 

1 998b).  Others  believe  the  risks  can  be  evaluated  using  a  fi-amework  based  on  business 
value  and  the  operational  performance  of  IT  (Earl,  1996).  Only  after  considering  such 
risks,  along  with  any  potential  gains  will  a  decision  on  outsourcing  be  arrived  at. 
Consequently,  each  proposed  determinant  factor  listed  below  has  both  benefit  and  risk 
determinants  associated  with  it.  Each  benefit  could  be  considered  to  have  a  positive 
influence  on  the  research  question,  while  a  risk  could  be  considered  as  having  a  negative 
effect.  However,  since  this  these  relationships  have  not  been  tested,  it  is  difficult  to 
confirm  their  existence. 

Based  on  this  information,  a  conceptual  model  for  IT  determinants  in  the  military 
was  developed.  As  shown  in  Figure  1,  the  five  categories  created  for  these  determinants 
are:  cost,  business,  technical,  political,  security,  and  environmental.  It  is  theorized  that 
each  of  these  factors  might  contribute  and  impact  an  outsourcing  decision.  For  the 
remainder  of  the  text,  the  terms  IT  outsourcing  and  outsourcing  will  be  used 
interchangeably  imless  specifically  differentiated. 
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Figure  1.  DoD  IT  Determinants  Outsourcing  Model 


Financial  Determinants 

Financial  determinants  are  those  factors  that  relate  to  the  fiscal  aspects  of  IT 
outsourcing.  They  directly  or  indirectly  impact  overall  costs.  Some  have  labeled  this 
factor  the  most  important  advantage  provided  by  outsourcing  (Rebeiro,  1996).  In  the 
DoD,  these  determinants  seem  to  take  on  special  meaning  since  research  has  shown  that 
government  agencies  focus  on  them  much  more  than  public  firms  (Jones,  1999).  The 
proposed  cost  sub-factors  are  listed  in  Figure  2. 
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Figure  2.  Cost  Determinants 


Benefits:  Financial 

Cost  Reductions.  Its  been  shown  that  one  of  the  largest  motivators  behind  the  use 
of  IT  outsotircing  in  the  government  is  reduced  costs  (Jones,  1999).  During  this  research, 
cost  reductions  were  repeatedly  labeled  the  number  one  reason  why  outsourcing  solutions 
should  be  pursued.  In  feet,  one  respondent  casually  remarked,  “is  there  another  reason  you 
would  outsourcer’  (Folsom,  2000).  A  government  Chief  Information  OfBcer  (CIO)  has 
even  been  quoted  that  when  it  comes  to  outsourcing,  “cost  is  THE  driver”  (Jones, 

1999:29).  Cost  reduction  mechanisms  might  include  economies  of  scale,  tighter  control 
over  fringe  benefits,  relocation  of  data  centers  to  lower  cost  areas,  and  more  fi)cused 
expertise  in  managing  IT.  (Antonucci,  1998;  Smith,  1998) 
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Improved  cost  controls.  Typically  IT  costs  arc  directly  related  to  user  demand. 
However,  most  organizations  do  not  have  the  ability  to  monitor  these  demands.  Therefore, 
many  firms  simply  use  general  allocation  systems  to  control  their  IT  distributions 
(Venkatraman,  1997).  Such  systems  do  nothing  to  encourage  users  not  to  request 
unnecessary  or  excessive  IT  resources.  However,  in  a  cost  controlled  environment,  users 
would  no  longer  be  able  to  call  their  favorite  analysts  to  request  unneeded  changes,  but 
instead  would  be  required  to  submit  requests  through  a  formal  cost  control  process. 
(Lacity,  1994) 

Restructuring  IT  budgets.  Sometimes  it  is  desirable  to  restructure  capital  budgets 
into  more  flexible  operating  budgets  (Lacity,  1994).  This  would  substitute  operations  and 
maintenance  charges  for  large  capital  expenditures.  As  an  example,  rather  than  purchasing 
large  computer  systems,  time  on  a  vendor’s  machine  could  be  purchased  on  an  as  needed 
basis.  For  the  private  sector  firm  this  helps  generate  cash  up  fi'ont,  which  increases  the 
participants’  cash  flow.  In  public  sector  firms  it  might  allow  different  appropriations  to  be 
used  for  computer  operations  (Barclay,  2000).  This  strategy  would  have  an  advantage 
when  operations  and  maintenance  (O&M)  funding  is  easier  to  obtain  than  procurement 
dollars. 

Protect  and  stabilize  IT  budgets.  As  government  budgets  continue  to  decline,  IT 
operations,  like  all  other  activities  within  the  government,  must  remain  efficient  and 
effective.  Consequently,  IT  budgets  are  routinely  inspected  for  any  savings.  Additionally, 
given  the  commodity  like  nature  of  IT  systems,  recapitalization  budgets  for  them  are 
programmed  well  in  advance.  These  large,  static  budgets  sometimes  make  an  easy  target 
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during  budget  reductions  exercises.  However,  if  these  operations  were  outsourced,  the 
funding  would  be  more  difficult  to  raid  due  in  part  to  contracting  regulations  (Barclay, 
2000).  Additionally,  long-term  contracts  improve  the  financial  predictability  associated 

with  these  requirements  (Martinsons,  1993). 

However,  this  strategy  can  backfire  if  the  contractor  attempts  to  extort  more 
funding  fi'om  the  customer  at  crucial  times.  This  is  easier  for  them  if  they  have  become 
fiiUy  entrenched  within  a  firm  (Quinn,  1999). 

Risk:  Financial 

Cost  Reductinn  T^isks  While  cost  savings  remain  one  of  the  more  attractive  lures 
of  outsourcing,  some  experts  remain  skeptical  about  the  actual  savings  it  provides.  As  an 
example,  some  believe  that  overall  costs  could  actually  increase  when  outsourcing  is  used 
since  certain  organizational,  coordination,  or  transactional  costs  usually  rise  when  it  is 
implemented,  especially  in  public  organizations  (Hancox,  1999).  Others  suggest  that  any 
benefits  outsourcing  actually  does  provide  are  slight.  They  explain  that  vendor  discounts 
achieved  fi’om  economies  of  scale  can  sometimes  be  neghgible  (Martinsons,  1993).  They 
further  argue  that  changes  in  software  Ucensing  agreements  have  greatly  reduced  a  vendors 
advantage  in  this  area  (Lacity,  1993).  Also,  since  costs  are  difficult  to  estimate,  inaccurate 
cost  saving  projections  may  be  made  (Quinn,  1999).  Even  when  IT  costs  can  be  estimated, 
vendors  will  sometimes  charge  for  services  that  a  customer  might  easily  assume  are 
covered  in  the  contract  such  as  supplies,  office  space  or  documentation.  The  vendor  might 
also  try  to  hide  the  true  costs  of  a  contract  by  providing  a  financial  package  whose  net 
present  value  is  extremely  attractive.  This  may  take  the  form  of  delayed  payments  or  credit 
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for  a  customer’s  hardware  (Lacity,  1993).  Some  argue  that  such  hidden  costs  are  the 
biggest  problem  associated  with  IT  outsourcing  (Willcocks,  1994).  It  has  been  proposed 
that  to  truly  determine  whether  outsourcing  wUl  produce  cost  savings,  senior  managers 
must  know  all  the  current  costs  of  their  information  management  activities  (Martinsons, 
1993). 

In  addition,  vendors  typically  cut  service  to  lower  costs  (Hirschheim,  2000).  It  has 
been  shown  that  if  internal  IT  departments  are  allowed  to  decreases  service  by  comparable 
amounts  they  could  also  attain  similar  cost  savings.  While  such  actions  might  appease 
senior  management,  users  typically  become  upset  when  there  IT  service  levels  decrease 
(Lacity,  1993). 

Business  Determinants 

Business  determinants  focus  on  elements  of  organizational  efficiency  and 
effectiveness.  It’s  been  proposed  that  when  these  factors  are  in  balance,  IT  can  be  folly 
exploited  (DiRomualdo,  1998).  As  seen  in  Figure  3,  benefits  associated  with  these  factors 
include  focus  on  core  competency,  influence  of  government  regulations,  and  faeilitating 
reorganizations.  Risks  include  the  inability  to  write  an  adequate  serviee  level  agreement, 
inability  to  manage  a  contract,  inability  to  folly  leverage  IT  and  loss  of  control. 
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Figure  3.  Business  Determinants 


Benefits:  Business 

Focus  on  core  competency.  During  the  1990’s  several  large  corporations 
abandoned  their  strategy  of  diversification  for  a  more  focused  approach.  Originally, 
diversification  was  used  to  mediate  risks  but  now  some  executives  beheve  that  a  strategic 
focus  is  a  more  important  competitive  advantage  (Lacity,  1994).  By  concentrating  on 
core  competencies  it  is  thought  that  a  company  can  focus  more  of  its  power  than  anyone 
else  on  the  few  capabilities  that  customers  genuinely  care  about  (Qumn,  1999).  Therefore, 
core  competencies  are  those  things  a  company  does  better  than  any  other.  They  can  be 
both  skills  and  systems  that  are  considered  to  operate  at  best  in  world  levels  and  “though 
which  a  company  creates  uniquely  high  value  for  customers”  (Quinn,  1999: 12).  As  an 
example,  both  Sears  and  General  Electric  have  recently  divested  themselves  of  many 
ancillary  business  units  to  focus  on  their  core  activities  (Rumsfled,  1995;  Torode,  1997). 
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Public  companies  have  adopted  this  idea  so  strongly;  some  have  listed  it  as  the  primary 
reason  to  outsource  (Jones,  1999). 

This  refocusing  has  several  effects  including  simplifying  the  management  agenda, 
flattening  organizations,  leveraging  innovation,  increased  employee  motivation  and 
cohesion,  and  freeing  up  resources  in  the  form  of  funding  and  personnel  to  apply  to  more 
strategic  activities  (Lacity,  1995;  Martinsons,  1993;  Quinn,  1999;  Smith,  1998).  Other 
benefits  in  this  area  include  increased  hardware  utilization,  improved  manufacturing  yield 
and  productivity  improvements  (Savage,  1998). 

However,  some  suggest  that  the  selection  of  IT  as  simply  core  or  non-core  is  too 
rudimentary.  They  would  argue  that  the  subject  needs  to  be  decomposed  even  further. 
Consequently,  frameworks  have  now  been  created  that  conqjare  the  contribution  of  an  IT 
activity  to  business  operations  against  the  contribution  of  an  IT  activity  to  business 
positioning  (Lacity,  1996).  However,  for  the  purposes  of  this  research,  that  distinction  will 
not  be  considered.  Only  the  simple  existence  of  the  core  competency  variable  will  be 
examined. 

Gnyemment  repnlatinns  and  guidelines.  Several  recent  Congressional  actions  such 
as  the  Government  Performance  and  Results  Act  of  1993  and  the  Paperwork  Reduction 
Act  of  1995  require  government  to  work  more  efficiently.  With  the  enactment  of  the 
Information  Technology  Management  Reform  Act  of  1996,  IT  is  looked  at  as  an  enabler  to 
make  this  happen.  However,  IT  itself  is  under  the  same  scrutiny  to  maintain  efficiency. 
Outsourcing  can  potentially  be  used  as  a  method  to  demonstrate  an  agencies  desire  to 
maintain  this  efficiency.  In  addition,  some  government  regulations  mandate  the  use  of 
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outsourcing  in  certain  situations.  Therefore,  IT  managers  might  be  required  to  outsource 
regardless  of  other  factors. 

F^rilitatinp  rcorpnniy^tions.  Modifications  to  organizational  structures  are  made 
for  a  variety  of  reasons.  Changes  in  mission,  tasks,  reporting  relationships,  and 
coordination  mechanisms  are  just  a  few  (Griffen,  2000).  Companies  can  use  outsourcing 
as  a  change  agent  when  they  want  to  dramaticaUy  alter  their  overaU  structure  (Scott, 

1995). 

After  reorganizations  occurs,  IT  managers  can  then  use  it  to  absorb  and  unite 
dissimilar  systems  into  their  firms  existing  IT  architecture.  In  the  private  sector  such 
actions  must  not  only  occur  when  a  reorganizations  occurs,  but  also  after  a  merger  or 
acquisition.  Therefore,  outsourcing  can  be  viewed  as  a  means  to  solve  technical 
inconqjatibilities,  absorb  excess  IT  assets  and  absorb  unnecessary  IT  employees  generated 
by  a  merger  (Lacity,  1994). 

Rbks:  Business 

Inability  to  write  an  adequate  service  level  agreement.  A  firms’  ability  to  establish  a 
solid  service  level  agreement  (SLA)  might  also  effect  its  decision  to  outsource  (Lacity, 

1995) .  SLA’s  provide  a  fiiU  and  detailed  description  of  the  services,  an  agreed  standard  of 
service,  a  service  level  appraisal  procedure  and  an  audit  provision  for  a  contract  (Rebeiro, 

1996) .  Some  feel  that  this  is  such  an  important  document  it  is  the  only  instrument  that  can 
ensure  expectations  will  be  realized  after  an  outsourcing  decision  has  been  made  (Klochko, 
1994).  However,  it  is  sometimes  very  difficult  to  precisely  specify  what  is  desired  (Quinn, 
1999).  To  develop  a  good  SLA,  a  firm  must  first  understand  what  services  are  needed  and 
then  translate  those  needs  into  specific  contract  deliverables.  Both  of  these  actions  present 
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several  problems.  First,  in  order  to  understand  what  services  a  firm  needs  it  must  first 
understand  its  own  processes.  Only  after  these  processes  are  fully  understood  can  the 
firms’  information  requirements  be  established.  Gaining  this  understanding  can  he  a  very 
difficult  task  (Hoffer,  1999).  Second,  once  these  requirements  are  established  they  must  he 
properly  transformed  into  binding  contractual  deliverables.  During  this  transformation, 
attention  to  detail  becomes  critical.  Recent  studies  have  shown  that  even  the  approach  and 
strategy  used  for  IT  outsourcing  can  affect  the  business  value  (Richmond,  1993).  In  fact, 
“the  price  the  user  pays  for  the  system  depends  on  the  form  of  the  contract  and  the  degree 
of  competition  among  the  vendors,  with  lower  prices  associated  with  more  competition” 
(Richmond,  1993:  71).  The  contract  needs  to  be  structmed  so  that  it  can  change  based  on 
what  is,  or  is  not,  known  about  the  “business,  the  course  of  technology,  and  the  capabifities 
of  outside  providers  and  the  conq)any’s  own  IT  department”  (Lacity,  1995:  89). 

However,  the  problems  associated  with  this  process  should  lessen  as  the  DoD 
moves  from  statements  of  work  (SOW)  to  performance  work  statements  (PWS),  which  is 
the  heart  of  the  A-76  process  (Paddock,  1987).  Whereas  a  SOW  might  list  detailed 
procedures  and  specifications  associated  with  completing  a  job,  a  PWS  only  provides  the 
general  requirement  and  lets  the  vendor  decide  how  to  perform  the  task  (Harney,  1998). 
This  is  a  more  dynamic  vehicle  since  it  allows  the  vendor  to  use  discretion  when  allocating 
their  resources  and  deciding  how  to  best  provide  the  services  required  (Paddock,  1987). 
Oftentimes  the  requested  services  are  abstractions,  such  as  pilot  training,  as  opposed  to 
flight  simulators,  as  would  be  the  case  in  a  statement  of  work  contract. 
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Further  flexibility  can  be  gained  by  agreeing  to  short-term  contracts  and  including 
clauses  that  mandate  periodic  reviews.  However,  these  items  do  not  come  cheap.  In 
reality,  they  are  usually  accompanied  by  cost  premiums.  Additionally,  any  variations  from 
the  original  requirements  must  always  be  funded.  Therefore,  some  believe  that  even  if  the 
requirements  could  be  accurately  captured  and  communicated  to  the  vendor,  any  savings 
would  be  negligible  after  they  were  calculated  (Earl,  1996) 

TnaKUity  tn  manflpe  a  contract.  Along  with  the  dangers  associated  with  developing 
an  outsourcing  contract,  there  are  several  risks  associated  with  a  firm  attempting  to  control 
it.  Typically,  members  of  the  internal  IT  staff  are  tasked  to  perform  this  function. 

However,  many  times  these  individuals  have  not  been  properly  trained  for  these  duties 
(Lacity,  1995).  They  may  be  excellent  programmers  or  computer  maintainers,  but  not  have 
the  necessary  to  oversee  and  monitor  contractual  arrangements.  This  change  can  be  a 
traumatic  experience  for  senior  IT  managers  who  have  to  modify  their  roles  from  handling 
personnel  issues  to  managing  contract  negotiations  and  administrative  functions.  As  Earl 
asks,  “if  the  IT  activity  has  been  badly  managed  in  the  &st  place,  will  the  IT  managers  be 
any  better  at  managing  an  external  provider?”  (Earl,  1996:  27).  Some  feel  that  it  is 
important  to  consider  a  firms  ability  to  perform  these  fimctions  before  an  outsourcing 
decision  is  made  (Lacity,  1995). 

Inability  to  fiillv  leverage  IT.  In  the  past  IT  was  simply  viewed  as  an  overhead 
burden  that  needed  to  be  funded.  It  was  considered  part  of  a  firm  s  infrastructure,  able  to 
only  support  other  more  value  added  functions  and  roles.  It  was  not  considered  an  integral 
part  of  a  firm  since  it  did  not  create  profit.  Since  IT  was  only  viewed  as  serving  non-core 
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functions,  it  was  subject  to  outsourcing  (Quinn,  1999).  However,  organizations  are  now 
starting  to  view  IT  as  an  investment  and  even  profit  center.  They  no  longer  consider  it  as 
they  do  office  supplies,  but  rather  are  starting  to  understand  that  IT  can  maximize  business 
opportunities  and  even  generate  revenue  (Venkatraman,  1997). 

Many  times,  a  firm’s  abihty  to  get  IT  to  perform  as  a  profit  center  is  based  on  its 
experience  with  it  (Hammer,  1994).  A  firm  must  work  closely  with  the  technology  to  truly 
understand  it.  In  support  of  this,  Hammer  and  Champy  believe  that  firms  need  to  start 
thinVing  about  IT  inductively  rather  than  deductively  (Hammer,  1994).  Only  after 
considering  what  IT  can  do  for  a  business  rather  than  how  IT  can  solve  a  specific  problem, 
can  the  true  potential  of  it  be  realized. 

However,  if  a  firm  outsources  its  IT,  there  is  less  potential  that  it  will  be  able  to 
realize  and  understand  its  true  power.  Many  times  marketable  IT  systems  were  discovered 
only  after  employees  recognized  that  systems  originally  intended  for  internal  use  had 
potential  in  the  marketplace.  It  is  possible  that  this  important  organizational  learning 
phenomenon  will  be  lost  if  IT  is  outsoureed  (Earl,  1996) 

T.nss  of  control.  Perhaps  the  greatest  disadvantage  to  an  outsourcing  company  is 
the  loss  of  direct  control  over  the  business  areas  outsoureed  (Rebeiro,  1996).  One  reason 
why  this  may  be  such  a  risk  are  all  the  potential  ways  control  can  be  lost.  For  example,  if  it 
is  difficult  for  customers  to  relate  IT  requirements  to  the  vendor,  the  vendor  may  simply 
provide  the  services  they  deem  appropriate  whieh  may  not  necessarily  be  what  the 
customer  wants.  Therefore,  the  customer  can  lose  eontrol  of  the  services  provided. 
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In  addition,  when  services  are  outsourced,  there  is  typically  no  longer  a  need  for  as 
many  internal  IT  specialists.  However,  if  these  people  leave  a  company  they  take  their 
skills  with  them.  When  this  occurs  the  firm  can  lose  control  over  their  own  destiny  by 
becoming  overly  dependent  on  the  contractor.  This  can  fuel  a  further  loss  of  control  over 
the  timing  and  quafity  of  outputs  (Quinn,  1999).  This  can  become  important  in  the  military 
when  battlefield  commanders  halfway  around  the  world  need  stateside  support,  but  are 
refused  because  a  contract  does  not  allow  for  it.  As  some  have  recognized,  IT  outsourcing 
is  nearly  an  irrevocable  act.  Once  it  is  performed  it  is  nearly  impossible  to  repeal. 
(Martinsons,  1993;  NS  A,  1999a). 

Technical  Determinants 

As  the  technology  that  drives  modem  information  systems  continues  to  evolve  at 
breakneck  speeds,  a  firm’s  ability  to  keep  up  becomes  even  more  difficult.  Companies 
want  to  ensure  that  the  hardware  and  software  resources  driving  their  IT  systems  are  as 
productive  as  possible  (DiRomualdo,  1998).  Consequently,  organizations  will  sometimes 
consider  outsourcing  as  a  way  to  stay  current  and  maintain  a  techmcal  edge  over  their 
competition.  As  can  be  seen  in  Figure  4,  in^roving  technical  services,  access  to  techmcal 
talent  and  technologies  are  all  outsourcing  benefits  that  organizations  hope  will  help  them 
maintain  their  edge.  To  some  in  government,  these  determinants  have  been  considered  the 
second  most  important  set  of  factors  after  reducing  costs  (Jones,  1999). 
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Decision  to 
Outsource 


TECHNICAL 

Benefits: 

-  Improving  Technical  Services 

-  Access  to  Technical  Talent 

-  Access  to  New  Technologies 
Risks: 

-  Problems  accessing  new  talent 

-  Problems  with  new  technologies 
-Inadequate  back-up  capability 

-  Excess  integration  problems 


Figure  4.  Technical  Determinants 


Benefits:  Technical 

Tmprrtvinp  technical  services.  Many  times  a  company  will  turn  to  outsourcing  when 


they  are  dissatisfied  with  their  own  IT  departments.  Systems  deUvered  late  and  over 


budget  as  well  as  delays  responding  to  user  requests  are  some  of  the  problems  that  cause 


this  attitude  (Lacity,  1994). 

Access  to  technical  talent.  As  technology  continues  to  become  more  complex  and 
specialized,  it  is  difficult  to  find  experts  that  understand  how  to  manage  it.  Further,  even  if 
a  company  does  succeed  in  luring  these  individuals,  many  are  finding  it  difficult  to  retain 
them.  (Lacity,  1994).  Some  consider  a  firm’s  ability  to  locate  and  acquire  these  individuals 

crucial  to  their  success  (Slaughter,  1996). 

By  acquiring  new  talent  via  outsourcing  a  firm  will  not  only  infiise  its  current  staff 
with  new  thoughts,  ideas  and  experiences,  but  also  gain  the  fi’eedom  to  reassign  their 
current  internal  staff  members  to  higher  priority  activities  such  as  the  development  of  more 
strategic-level  systems  that  can  deliver  a  competitive  advantage.  Many  times  this  personnel 
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redistribution  can  increase  productivity  and  create  a  stronger  foundation  for  fiiture  business 
success  (Martinsons,  1993).  This  advantage  can  be  considerable  in  the  military  since  it 
costs  more  to  develop  an  internal,  skilled  IT  worker  than  it  does  in  the  private  sector  due 
to  the  additional  military  training  a  member  must  undergo.  Consequently,  it  is  especially 
important  to  assign  these  individuals  to  the  systems  and  programs  were  the  return  on 
investment  is  the  highest  possible. 

Access  to  new  technologies.  Many  times  outsourcing  is  seen  as  a  tool  that  can 
provide  a  firm  with  emerging  technology  by  giving  them  access  to  a  vendors  large  research 
and  development  efforts.  They  feel  that  the  vendors  can  help  them  leapfi-og  or  catch  up  to 
the  competition  (Martinsons,  1993).  Hardware  and  software  systems  today  need  to  be 
constantly  updated  or  replaced.  It  is  grovring  increasingly  difficult  to  stay  current  given  the 
rate  of  change  (Antonucci,  1998). 

Risks:  Technical 

Prttftntifll  problems  with  obtaining  new  technical  talent.  As  one  might  expect, 
there  are  some  who  believe  that  the  expectation  outsourcing  will  provide  new  talent  to  an 
organization  is  overstated.  They  point  out  that  as  organizations  move  towards 
outsourcing,  employees  start  fearing  they  will  lose  their  jobs.  To  appease  this  fear  agencies 
start  considering  different  vertical  restraints  in  the  principle-agent  relationship,  such  as  soft- 
sourcing.  Soft-sourcing  is  when  a  customers  employees  are  given  preferential  treatment 
when  applying  for  their  job  after  it  has  been  outsourced  (NS  A,  1999b).  In  feet,  this 
provision  can  even  be  included  as  a  clause  to  an  A-76  study  or  an  outsourcing  contract. 
While  such  a  clause  might  help  minimize  any  personal  impact  caused  by  outsourcing,  it  can 
coimteract  the  desired  effect  of  introducing  new  technical  talent  into  the  organization. 
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AdditionaUy,  any  above  average  talent  hired  by  a  vendor  many  times  will  not  even  be 
assigned  to  their  original  jobs.  Instead,  the  vendor  will  use  them  to  win  new  contracts  and 
accounts  (Lacity,  1994). 

Also,  many  new  con^anies  that  compete  for  IT  contracts  do  not  have  a  large  pool 
of  talent.  What  qualified  employees  they  might  have  only  posses  a  small  amount  of 
experience  solving  the  specific  type  of  problems  their  previous  customers  had.  Also,  how 
can  a  customer  be  certain  that  a  vendor  will  continue  to  ujjdate  and  keep  their  staff 
current?  Many  times,  outsourcing  firms  will  win  a  contract  to  manage  a  new  technology 
but  eventually  even  this  technology  will  become  obsolete.  Customers  continually  want  the 
latest  technology  to  increase  productivity,  however,  if  the  vendor  does  not  upgrade  their 
staff,  it  will  be  difficult  for  them  to  respond  (Earl,  1996). 

Potential  pmhlpims  with  accessing  new  technologies.  As  systems  become  more 
advanced,  they  also  become  more  complex  and  interdependent.  This  makes  it  increasing 
difficult  to  isolate  problems  when  they  arise.  It  is  sometimes  difficult  to  determine  if  a 
problem  is  caused  by  a  terminal,  network,  application,  or  communication  system.  If 
outsourcing  is  used,  vendor  and  customer  interfaces  only  serve  to  complicate  this  problem 
(Earl,  1996). 

Also,  some  argue  that  the  newer  a  technology  is,  the  less  it  should  be  outsourced. 
They  befieve  a  firm  will  face  additional  risk  when  they  outsource  immature  technologies 
because  it  is  more  difficult  to  adequately  capture  contract  requirements  due  to  a  lack  of 
understanding  surrounding  it  (Lacity,  1996).  These  individuals  would  therefore  argue  that 
mature  technologies  are  easier  to  outsource  since  the  firm  has  usually  overcome  the  initial 
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learning  curve  and  reached  a  point  were  it  now  understands  the  technology  and  is  able  to 
adequately  define  requirements. 

Tnarlequate  recoverv  and  back-up  capability.  Several  computer  dependent 
organizations  have  gone  into  liquidation  follo>ving  system  disasters  (Fink,  1994). 
Consequently,  it  is  important  that  organizations  ensure  they  are  protected  fi’om  such  blows 
or  can  recover  fi-om  them  should  they  occur.  If  a  firm’s  management  does  not  feel  that  a 
vendor  could  adequately  guarantee  the  protection  of  its  information,  it  might  be  less  likely 
to  outsource. 

Excess  intppT?^tinn  and  interoperability  problems.  When  considering  whether  to 
outsource  IT,  integration  issues  can  also  become  a  major  concern  (Lacity,  1996).  When 
the  required  amoxmt  of  integration  between  an  outsourced  activity  and  other  business 
processes  or  technical  systems  is  high,  the  risks  associated  with  outsourcing  increase.  If 
two  different  contractors  control  separate  but  interrelated  activities,  finger  pointing  can 
arise  between  them  when  the  entire  system  is  not  working  correctly.  This  leaves  the 
organization  caught  in  the  middle  with  little  information  as  to  where  the  real  problem  lies. 

Some  suggest  that  this  problem  is  even  more  of  an  issue  in  the  government  due  to 
its  passion  with  interoperabiUty.  Some  federal  CIO’s  have  even  “acknowledged  that  their 
number  one  priority  is  interoperability  among  the  division  in  the  enterprise  (Jones,  1999). 

Political  Determinants 

As  one  might  expect  the  political  aspects  of  outsourcing  might  be  most  powerful 
within  a  public  sector  firm.  The  political  dimension  involves  “the  behavior  of  the  various 
parties  involved  in  the  decision-making  process  and  how  they  shape  senior  management’s 
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perception  about  IT  and  its  value”  (Lacity,  1994).  For  the  purposes  of  this  research  these 
determinants  focus  on  the  satisfaction  of  personal  needs.  As  seen  in  Figure  5,  there  are 
several  potential  political  determinants  including:  proving  efficiency,  justifying  new 
resources,  oqjosing  exaggerated  claims,  eliminating  a  troublesome  function,  breaking  the 
so-called  glass  ceiling  and  lack  of  trust. 


Figure  5.  Political  Determinants 


Benefits:  Political 

Proving  efficiency.  Many  times  organizations  will  account  for  IT  expenditures 
simply  as  an  overhead  expense  and  evaluate  them  based  on  cost  efficiency.  When  this  is 
done  however,  it  is  difficult  to  develop  any  concrete  measures  of  efficiency.  Therefore, 
some  IT  managers  will  support  obtaining  outsourcing  proposals  in  the  hopes  that  it  will 
prove  the  efficiency  of  the  internal  IT  departments  and  justify  its  continued  existence.  It 
has  been  shown  that  after  an  outsourcing  evaluation  was  conducted  IT  managers  have 
successful  convinced  senior  managers  and  users  that  their  internal  IT  departments  could 
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perform  the  same  services  an  outside  vendor  could  perform  for  the  same  amount  or  less 
(Hirschheim,  2000). 

A  potential  drawback  to  this  strategy  is  the  chance  senior  managers  might  view  the 
results  with  skepticism.  This  could  occur  if  they  believe  that  IT  managers  biased  the  results 
by  selectively  picking  only  their  most  efficient  and  best  managed  projects  for  the 
outsourcing  evaluation.  If  they  did  not  include  an  accurate  representation  of  their  IT 
department  in  the  request  for  proposal,  the  results  would  not  be  a  true  reflection  of  the 

costs  associated  with  an  outsourcing  solution. 

Tii«;titvinp  new  resources.  Since  IT  is  many  times  looked  upon  only  as  a  cost 
center,  senior  managers  will  proceed  warily  when  asked  to  dedicate  additional  resources  to 
it  (Venkatraman,  1997).  Therefore,  IT  managers  believe  that  if  they  can  show  outsourcing 
is  not  a  more  cost  effective  and  efficient  method  to  obtain  new  services,  senior  leaders  will 
be  more  inclined  to  authorize  additional  funding  for  internal  projects.  (Lacity,  1994) 

However,  this  request  may  also  be  considered  biased  if  senior  management  does 
not  view  the  justifications  as  credible.  Some  believe  that  the  best  way  to  handle  a 
credibility  issue  within  an  IT  department  is  to  involve  senior  management,  and  outside 
independent  experts,  in  the  outsourcing  evaluation.  (Lacity,  1994) 

FYpnsinp  <»vapperated  claims.  With  the  explosion  in  IT  outsourcing,  many  senior 
managers  are  anxious  to  ensure  that  their  firms  are  taking  full  advantage  of  this  new 
method  of  IT  governance  (Loh,  1992a).  However,  IT  managers  sometimes  fear  this  zeal 
will  seduce  their  leaders  into  an  outsourcii^  relationship  prematurely.  Consequently,  some 
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IT  mangers  will  commission  outsourcing  evaluations  to  expose  any  exaggerated  claims  an 
outside  firm  has  made  (Lacity,  1994). 

However,  if  senior  management  does  not  feel  the  IT  department  is  capable  of 
making  an  unbiased,  rational  outsourcing  decision,  they  may  commission  their  own 
outsourcing  evaluation  without  the  involvement  of  the  IT  department.  Therefore,  if  senior 
managers  believe  that  IT  managers  are  stalling  on  an  outsourcing  decision  by  constantly 
researching  the  problem,  they  may  move  ahead  with  the  decision  independent  of  the  IT 
group.  (Lacity,  1994) 

Fliminatinp  a  troublesome  function.  Due  to  the  seemingly  constant  stream  of 
available  upgrades  for  information  systems,  its  growing  increasingly  difficult  for  a  firm’s  IT 
and  senior  mangers  to  understand  how  newer  systems  can  be  leveraged  to  make  the 
organization  run  more  efficiently.  In  addition,  IT  mangers  and  staff  members  usually 
receive  few  praises  but  much  abuse.  When  the  systems  are  working  correctly  few  people 
even  notice  the  actions  of  these  individuals.  However,  when  something  goes  Avrong, 
everyone  immediately  starts  to  curse  and  disparage  them.  This  has  caused  several 
individuals  to  re-evaluate  whether  the  constant  flow  of  problems  associated  with  the  work 

is  truly  worth  any  compensation  (Lacity,  1994). 

Breaking  the  so-called  glass  cfidirig.  Many  IT  directors  see  themselves  as  not  being 
fully  appreciated  and  valued  by  senior  management.  Few  IT  managers  break  into  the  upper 
echelons  of  management  within  a  firm  (Lacity,  1994).  Therefore,  they  might  turn  to 
outsourcing  as  a  way  to  appear  more  corporate  and  creditable  in  the  hopes  of  enhancing 
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their  own  career.  Research  has  shown  that  several  IT  managers  have  initiated  outsourcing 
evaluations  to  help  alleviate  any  misconceptions  that  they  are  myopic  (Lacity,  1994). 

Risks:  Political 

T.ack  of  trust  between  government  and  contractors.  It  is  important  that  in  any 
contracting  action,  trust  exists  between  the  principal  and  agent.  Trust  can  be  defined  as  a 
confidence  that  the  behavior  of  another  will  conform  to  one’s  expectations  and  in  the 
goodwill  of  another”  (Hart,  1997:24).  When  IT  work  is  performed  in-house  the 
participants  typically  know  one  another  and  therefore  have  a  certain  degree  of  trust  in  each 
other  (Sabherwal,  1999).  In  this  situation,  unexpected  problems  associated  with  the  work 
are  easier  to  deal  with.  However,  due  to  project  center  elements,  structural  mechanisms 
and  unique  reporting  arrangements,  trust  is  sometimes  more  difficult  to  obtain  in  IT 
outsourcing  arrangements.  Trust  stems  fi-om  a  psychological  contract  between  the 
principal  and  agent.  This  contract  consists  of  unwritten  expectations  held  by  both  parties 
regarding  each  other’s  prerogatives  and  obligations.  When  trust  is  present  a  “virtuous 
cycle”  (Sabherwal,  1999:  83)  is  present  between  the  psychological  contract,  the  written 
contract  and  performance.  This  leads  to  a  high  quality  product  in  a  timely  manner.  If  trust 
is  not  present  a  “vicious  cycle”  (Sabherwal,  1999:  83)  can  exist  between  the  same  three 
elements  and  lead  to  a  poor  quality  product  and  multiple  delays. 

Security  Determinants 

Security  is  a  fector  that  some  might  think  could  be  placed  in  the  business,  techmcal, 
or  even  political  category.  However,  due  to  several  unique  features  associated  with  the 
subject,  it  was  thought  that  making  it  a  separate  category  was  warranted.  Additionally, 
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security,  and  security  implications,  were  a  common  theme  and  source  of  uncertainty  for 
individuals  interviewed  during  the  development  of  the  conceptual  model. 

Security  issues  are  conunon  to  all  IT  contracts;  however,  in  outsourcing 
agreements  they  hold  special  meaning.  (Rebeiro,  1996).  As  an  example,  some  have 
proposed  that  the  security  framework  used  when  IT  work  is  outsourced  is  different  than  if 
the  work  is  kept  in-house.  The  key  reasons  given  for  this  are  a  shift  in  emphasis  from 
physical  protection  to  recovery,  changes  in  the  roles  of  application  controls,  and  the  need 
to  place  a  greater  reliance  on  general  controls  (Fink,  1994).  Also,  others  feel  that  a 
different  organizational  security  structure  is  needed  when  IT  work  is  outsourced 
(Sherwood,  1997).  It  is  important  that  both  the  customer  and  vendor  create  appropriate 
jobs  to  deal  with  potential  security  issues.  It  is  equally  important  to  formalize 
communication  channels  between  the  parties.  Commumcation  is  repeatedly  listed  as  the 
principal  factor  necessary  to  create  and  maintain  a  healthy  security  IT  outsourcing 
relationship  (Caldwell,  1997;  Sherwood,  1997;  Ryan,  1999). 

Another  feature  unique  to  the  security  of  DoD  systems  is  the  critical  nature  and 
special  importance  of  the  national  security  data  contained  in  them.  Little  doubt  exists  that 
the  DoD  has  special  security  requirements.  These  requirements  are  to  maintain  national 
security  and  ensure  mission  accomplishment  (DoAF,  1996;  DoD,  1996).  Consequently, 
any  information  contained  in  a  classified  IT  system  must  be  controlled  as  stringently  as  if  it 
were  in  hardcopy  form.  In  fact,  more  precautions  must  sometimes  be  taken  when  dealing 
with  classified  IT  systems  due  to  the  inherent  vulnerabilities  of  a  computer  system  that  do 
not  exist  for  hardcopy  documents  such  as  emanations  and  mass  storage  capability 
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(Denning,  1999).  Therefore,  it  is  no  surprise  that  government  agencies  invest  more  in 
security  than  public  firms  (Jones,  1999). 

However,  several  commercial  industries  such  as  banking,  telecommunications,  and 
online  commerce  also  have  very  high  security  requirements.  It  then  becomes  important  to 
determine  how  additionally  significant  security  is  to  DoD  outsourcing  decisions?  As  has 
been  asked  before,  “is  the  need  for  enhanced  security  for  military  operations  a  factor  to 
maintain  a  blue  suit  or  in-house  IT  capability  and  are  military  security  requirements  greater 
than  those  in  the  commercial  sector?”  (McTeman,  1997). 

Due  to  the  nature  of  security,  it  is  something  that  can  only  be  dimimshed  or  lost. 

The  greatest  amoimt  is  achieved  when  only  one  person  knows  the  information  and  does  not 
tell  anyone  else.  In  fact,  in  this  situation,  very  little  if  any  security  is  required.  However,  as 
the  number  of  people  with  access  to  the  information  grows,  so  does  the  need  for  security. 
As  the  amount  of  required  security  increases,  primary  work  process  tend  to  slow  (Gips, 
2000).  Security  checks,  special  procedures,  and  locked  rooms  all  impede  work 
requirements.  Additionally,  when  extra  processes  are  added  to  any  situation,  the  chances 
of  something  going  wrong  increase.  In  the  military,  when  something  goes  wrong  with 
security,  the  work  is  usually  halted.  Consequently,  it  is  difficult  to  conceive  of  a  situation 
where  security  is  a  benefit;  it  will  almost  always  be  a  risk. 

As  can  be  seen  in  Figure  6,  the  three  proposed  security  risks  are,  controlling  access 
to  the  information,  overall  vendor  security  procedures,  and  release  of  competitive 
advantage. 
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Figure  6.  Security  Determinants 

Risks:  Security 

Controllinp  access  to  the  information.  One  type  of  security  risk  concerns  a 
vendor’s  employees  and  the  access  they  have  to  information.  Typically,  computer 
programmers  and  maintenance  personnel  from  an  outsourcing  firm  require  detailed 
knowledge  of  a  customers  IT  infrastructure  to  do  their  job.  However,  by  granting  this 
level  of  access,  a  fair  amount  of  network  control  is  being  given  to  them  (Talley,  1998). 

Even  when  something  as  innocuous  as  secretarial  and  administrative  fimctions  are 
outsourced,  the  data  housed  in  a  customers  system  are  put  at  risk.  This  risk  is  even  more 
pronounced  when  external  security  specialists  are  used  to  inspect  and  audit  a  customers 

system. 

Given  the  rate  of  change  in  IT  along  with  the  size  and  complexity  of  many 
corporate  systems,  its  difficult  for  internal  computer  security  personnel  to  ensure  that  every 
network  is  secure.  Consequently,  many  large  companies  are  turning  to  external  security 
personnel  to  diagnose  and  test  their  systems  (Caton,  1999;  Talley,  1998).  Yet  these 
consultants  typicaUy  need  frill  and  complete  access  to  an  organizations  system  when 
conducting  an  audit.  This  helps  ensure  that  the  results  are  as  comprehensive  as  possible. 
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However,  since  an  external  firm  will  be  analyzing  the  vulnerabilities  of  a  customers 
network,  they  could  potentially  use  that  information  against  them.  If  the  DoD  were  the 
customer,  it  would  be  very  easy  for  the  consultant  to  seU  any  information  it  coUected  to  a 
potential  adversary  (Martinsons,  1993;  Quinn,  1999).  AdditionaUy,  since  background 
checks  in  the  private  sector  are  sometimes  not  as  thorough  as  they  could  be,  foreign  agents 
might  be  able  to  infiltrate  a  critical  DoD  IT  system  simply  by  posing  as  an  employee  of  the 
consulting  firm. 

Another  issue  relating  to  network  access  revolves  around  double  sourcing.  This  is 
the  practice  of  a  prime  contractor  using  subcontractors  to  fulfill  customer  requirements 
(Caldwell,  1997).  Some  experts  believe  that  an  average  of  36%  of  all  work  performed  on 
IT  contracts  is  done  using  double  sourcing  (Caldwell,  1997).  However,  since  most 
subcontractors  are  typically  not  as  large  as  prime  contractors,  it  is  harder  for  them  to 
perform  in  depth  background  checks  on  each  employee.  Consequently,  someone  who 
might  not  pass  the  original  customers  security  check  could  still  end  up  working  on  their  IT 
systems  without  them  ever  knowing.  One  potential  solution  to  this  problem  is  to  provide 
the  prime  contractor  with  a  list  of  pre-approved  vendors.  In  the  DoD,  this  practice  is 
known  as  using  directed  subcontraetors.  By  using  direeted  subcontractors,  the  DoD  might 
decrease  its  exposure  to  any  potential  security  problems  caused  by  an  unfamiliar  firm.  The 
problem  with  this  arrangement  is  that  the  DoD  assumes  most  risks  associated  with  that 
subcontractor.  Therefore,  if  the  subcontractor  defaults  for  any  reason,  the  Government 
does  not  have  recourse  back  to  the  prime  contractor.  Consequently,  any  cost  or  schedule 
impacts  caused  by  such  problems  would  be  the  sole  burden  of  the  DoD. 
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nvp;ra11  vendor  security  procedures.  Another  risk  is  a  potentially  weak  vendor 
security  plan.  How  vendors  handle  customer  data  on  their  own  systems,  correspond  with 
each  other  via  e-mail,  and  simply  dispose  of  a  customers  information  after  it  is  no  longer 
needed,  are  questions  that  might  need  to  be  considered  by  a  firm  before  it  decides  to 
contract  with  a  vendor  (Ryan,  1999).  In  addition,  vendors  typically  supply  services  to 
several  companies  including  competitors  and  adversaries.  Given  the  critical  nature  of  the 
national  security  information  contained  in  DoD  IT  systems,  the  benefits  of  outsourcing  may 
not  outweigh  the  risks. 

Release  of  competitive  advantage.  In  today’s  militaiy,  technology  plays  a  key  role. 
Our  forces  can  dominate  the  battlespace  because  of  their  access  to  the  latest  technology 
and  equipment.  However,  this  advantage  assumes  our  systems  are  more  advanced  than  the 
enemy’s.  To  ensure  the  United  States  preserves  this  advantage,  the  DoD  fimds  research 
and  development  projects  so  that  its  future  systems  will  always  be  more  current  and 
capable  than  those  of  its  adversaries.  Currently,  the  military  is  not  only  using  IT  to 
transmit  information,  but  also  to  deny,  disrupt,  disable  or  even  destroy  an  enemy  s 
infrastructure  (CJCS,  1996).  In  a  similar  manner,  corporations  are  starting  to  understand 
the  benefit  of  using  IT  systems  to  gain  competitive  advantages.  No  longer  are  they  viewed 
as  cost  centers.  Today,  IT  is  being  looked  at  as  a  profit  maker  (Venkatraman,  1997). 

However,  in  both  of  these  situations,  if  outsourcing  were  used,  there  is  the  potential 
that  the  technical  data  associated  with  these  systems  could  become  compromised.  If  this 
occurred,  an  adversary  might  be  able  to  determine  how  to  defeat  such  a  system,  or  build  a 
similar  system.  This  would  negate  any  advantage  (Maitinsons,  1993).  The  potential 


40 


F 


information  released  in  this  determinant  differs  from  the  information  that  might  be 
compromised  in  the  controlling  access  to  the  information  determinant  since  that  factor 
considers  only  the  information  contained  in  the  system,  whereas  this  determinant  focuses 
on  the  release  of  system  design  information.  Typically,  even  security  specialists  will  not 
have  access  to  this  type  of  information,  only  designers  or  specialized  maintenance 
personnel. 

Environmental  Determinants 

Like  security,  environmental  factors  could  be  considered  part  of  another  categories. 
However,  none  of  the  previously  mentioned  factors  captures  the  multifaceted  nature  of 
environmental  determinants.  As  an  example,  difiusion  theory  has  been  advanced  as  a 
mechanism  for  the  increased  use  of  IT  outsourcing.  This  theory  has  business  aspects 
attached  to  it  such  as  maintainmg  efficiency  and  effectiveness  as  well  as  political  overtones 
such  as  making  a  business  choice  based  on  the  behavior  of  others. 

Since  there  is  a  reduced  emphasis  placed  on  producing  a  profit  within  a  public 
sector  organization  (Hancox,  1999)  many  of  the  reasons  why  private  sector  firms 
outsource  IT  are  minimized  (Smith,  1999;  Loh,  1992a).  Therefore,  given  the  poUtical 
nature  of  pubUc  sector  firms,  environmental  factors  may  be  even  more  important  than  in 
the  private  sector.  As  seen  in  Figure  7,  examples  include  length  of  service,  imitative 
behavior  among  firms,  and  internal  resistance. 
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ENVIRONMENT 

Benefits: 

-  Length  of  service 

-  Changes  in  amount  of  outsourcing 
Risks: 

-  Internal  resistance 


Decision  to 
Outsource 


Figure  7.  Environment  Determinants 

Benefits:  Enviornmental 

T.ength  of  service  remaining  for  outsource  decision  makers.  Using  agency  theory 
it’s  been  shown  that  the  amount  of  time  IT  executives  feel  they  will  be  in  a  particular 
position  or  role  can  potentially  effect  the  outcome  of  an  outsourcing  decision  (Loh,  1993). 
The  time  horizon  is  “a  crucial  dimension  affecting  managerial  decision  making  with  respect 
to  IT  projects”  (Loh,  1993:223). 

Therefore,  the  amount  of  outsourcing  in  DoD  organizations  might  be  dependent  on 
the  fact  that  Military  members  fill  a  particular  position  only  temporarily.  As  programs 


continue  to  grow  in  complexity,  scope,  and  time  required  for  completion,  it  is  inevitable 
that  individuals  associated  with  such  programs  begin  to  realize  that  their  position  will  have 
a  successor.  This  might  help  an  incumbent  feel  less  responsible  for  their  actions  and  any 
potential  derogatory  outcomes.  It  is  therefore  conceivable  that  if  they  feel  they  will  be 
changing  jobs  in  the  near  fiiture,  they  will  be  more  likely  to  outsource.  On  the  other  hand, 
it  is  equally  possible  that  the  longer  they  know  they  will  be  in  a  position  the  more  likely 
they  will  outsource.  This  could  be  due  in  part  to  their  desire  to  rid  themselves  of  what  they 
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might  consider  the  small  repetitive  problems  IT  operations  present  and  focus  their  effort  on 
more  strategic  related  issues. 

Changes  in  the  amount  of  IT  outsourcing  within  the  DoD.  In  1989  Kodak 
outsourced  most  of  its  IT  infrastructure  requirements  to  IBM.  When  this  occurred,  some 
felt  a  great  deal  of  interest  was  generated  in  the  business  community  regarding  the  potential 
competitive  advantage  that  such  a  move  would  create  (Loh,  1992b). 

This  advantage  stems  from  the  feet  that  IT  outsourcing  can  be  considered  a 
fundamental  shift  in  the  business  strategy  (Henderson,  1992).  Specifically  it  represents  a 
significant  shift  in  the  IT  strategy  and  mode  of  governance.  IT  governance  includes  the 
choices  of  structural  mechanism  (e.g.,  joint  ventures  long-term  contracts,  equity 
partnerships,  joint  R&D)  a  firm  makes  to  obtain  the  required  IT  capabilities,  involving 
issues  such  as  the  deployment  of  proprietary  versus  common  networks  as  well  as  strategic 
choices  pertaining  to  development  of  partnerships  to  eiqiloit  IT  capabilities  and  services. 

In  addition,  it  radically  changes  the  internal  processes  and  routines  of  the  user  organization. 
As  such  it  can  be  considered  as  an  administrative  innovation.  Once  a  firm  adopts  an 
innovation,  it  is  difiused  or  communicated  through  certain  channels  over  time  among  the 
members  of  the  firms  social  system. 

Studies  have  shown  that  when  Kodak,  a  respected  firm  within  a  social  system, 
outsourced  its  IT  systems,  other  firms  in  the  system  followed  (Loh,  1992b).  This  has  been 
labeled  the  “Kodak  effect.”  This  might  explain  why  some  firms  continue  to  outsource  even 
after  they  have  been  warned  against  it.  Evidence  supports  that  this  diffusion  of  innovation 
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among  government  agencies  might  be  one  of  the  strongest  determinants  behind  IT 
outsourcing  (Kim,  1998). 

However,  others  feel  that  the  evidence  does  not  support  the  “Kodak  effect  (Hu, 
1997).  They  feel  the  parameters  used  to  prove  the  existence  of  the  effect  where  not 
properly  estimated  or  implemented.  However,  like  the  original  study,  they  also  claim  that 
the  diffusion  of  IT  outsourcing  is  not  a  random  process.  Yet,  they  feel  a  mixed  influence 
model  rather  than  the  internal-influence  model  used  in  the  first  study  best  describes  the 

process. 

Risks:  Environmental 

Internal  resistance.  Concern  over  displaced  IT  staff  members  can  effect  a  firms 
decision  to  outsource  its  IT  (Lacity,  1995).  Many  times  a  backlash  of  fear  among  IT  staff 
members  is  generated  when  outsourcing  is  considered  (Quinn,  1999).  When  IT  employees 
hear  their  jobs  are  in  jeopardy  of  being  outsourced  they  may  feel  demoralized  and  decide  to 
leave  a  company  (Antonucci,  1998).  Even  if  they  stay  and  are  not  laid-off,  they  may  feel 
that  outsourcing  has  decreased  the  visibility  of  their  jobs  and  decide  to  quit  (Martinsons, 
1993).  It  could  be  very  important  for  a  firm  to  consider  these  thoughts  before  they 
outsource.  Such  a  decision  may  cause  serious,  unwanted  repercussions. 

This  conceptual  model  represents  an  attempt  to  capture  aU  the  reasons  why  a  DoD 
or  IC  agency  might  outsource  their  IT.  It  was  developed  using  interviews  and  an  extensive 
literature  search.  The  next  chapter  will  focus  on  the  process  that  was  used  to  validate  it 
and  convert  it  into  more  of  an  analytic  form. 
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III.  Methodology 


Introduction 

As  shown,  there  are  a  number  of  potential  determinants  for  IT  outsourcing  in  the 
DoD,  which  help  form  a  conceptual  model.  The  goal  in  this  stage  of  the  research  was  to 
inductively  determine,  using  quantitative  measures,  the  validity  of  this  conceptual  model 
by  attempting  to  form  a  consensus  among  experts  in  several  Government  agencies 
regarding  the  validity  of  the  determinates  in  each  factor.  The  result  was  an  analytic  model 
that  can  be  analyzed  used  in  further  research. 

Development  of  the  Conceptual  Model 

Development  of  the  conceptual  model  began  by  interviewing  several  individuals 
working  in  IT  outsourcing  within  the  DoD.  These  interviews  lead  to  a  number  of  possible 
determinants,  which  were  then  used  to  initially  guide  a  Uterature  search.  This  search  was 
then  ejqianded  to  include  as  many  aspects  of  IT  outsourcing  as  possible  m  the  hopes  of 
providing  additional  insight  into  what  IT  outsourcing  determinants  are  present  in  the  DoD. 
Since  most  of  the  literature  pertained  to  private  sector  firms,  it  was  obvious  that  some 
determinants,  such  as  debt  and  equity  financing  (Loh,  1992a),  had  no  place  m  an 
outsourcing  theory  centered  on  pubHc  organizations.  Therefore,  such  determinants  were 

ultimately  dropped  firom  further  consideration. 

While  most  of  the  literature  used  to  develop  the  conceptual  model  was  based  on 
validated  findings,  some  was  notional  information  based  on  the  experiences  of  different 
authors.  Consequently,  the  final  conceptual  model  could  be  considered  as  a  combination 
of  expert  and  academic  opinion  along  with  grounded  theories.  Since  anecdotal  evidence 
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comprised  part  of  the  theory,  the  entire  model  needed  to  be  validated.  Once  this  was 
completed  it  could  be  considered  to  be  in  analytic  form. 

The  Delphi  Technique 

The  Delphi  technique  was  chosen  to  validate  the  conceptual  model  since  it  is  well 
suited  to  developing  a  consensus  of  expert  opinion.  The  basic  premise  behind  this  method 
is  that  two  (or  many)  minds  are  better  than  one.  It  was  originally  developed  by  the  RAND 
Corporation  in  the  1960’s  to  help  decision  makers  formulate  broad  or  long-range  policy. 

It  is  a  formalized  process  to  determine  the  best  answer  or  solution  to  a  problem  whenever 
insufficient  or  no  applicable  data  exists,  the  required  data  is  too  expensive  to  obtain  and 
analyze,  or  the  problem  variables  and  their  interaction  are  not  clearly  known. 

Consequently,  it  is  intended  to  work  in  the  regime  of  opinion  between  mere  speculation 
and  knowledge.  It  should  be  noted  that  while  the  traditional  purpose  of  the  Delphi 
method  is  to  make  predictions  about  the  future,  this  study  uses  it  to  define  critical  issues  in 
information  technology.  However,  its  use  for  this  purpose  is  weU  supported  (Dickson, 
1984;  Brancheau,  1987;  Niederman,  1991;  Brancheau,  1996;  Westbrook,  1997). 

Basically,  the  Delphi  approach  “solicits  input  from  a  panel  of  experts  who 
contribute  individually”  (Griffin,  1999:280).  It  is  based  on  an  iterative  set  of 
questionnaires  that  attempt  to  capture  the  thoughts  and  feelings  of  a  group  on  a  particular 
subject.  At  the  end  of  each  round,  input  obtained  firom  the  eiqierts  is  averaged  and  the 
results  broadcast  back  to  the  group.  The  same  questions  are  then  asked  again,  the  results 
analyzed,  and  the  conclusions  modified.  This  cycle  of  questions,  feedback,  questions 
continue  until  the  prediction  stabilizes.  The  final  conclusion  is  considered  to  be  the 
decision  of  the  experts.  (Clayton,  1997) 
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The  Delphi  method  can  be  considered  a  complement  to  the  panel  approach. 
However,  unlike  the  panel,  it  provides  more  information  regarding  “uncertamties  or 
disagreements  about  the  subject  and  quantitatively  evaluates  the  degree  of  uncertainty 
which  exists  within  a  large  group  of  experts.”  (Goldstein,  1971:225). 

Advantages  of  Delphi 

Anonymity  and  controlled  feedback. 

A  notable  characteristic  of  the  Delphi  method  is  that  it  masks  the  identities  of  the 
participants,  which  helps  lower  the  amount  of  group  discussion  (Tersine,  1976).  This  is 
desirable  since  it  has  been  shown  that  such  discussion  tends  to  produce  less  reUable  results 
than  when  contact  between  participants  is  controlled  (Dalkey,  1969). 

Less  ConfrontationaL 

Since  the  identities  of  the  participants  are  not  revealed,  the  input  provided  by  a 
specific  group  member  is  not  directly  linked  to  them.  All  feedback  given  is  an  aggregate 
of  member  responses.  This  allows  other  group  members  to  respond  to  comments  without 
regard  to  their  personal  feelings  about  the  originator.  In  addition,  since  interaction  is 
anonymous,  members  can  feel  free  to  change  their  minds  without  feeling  self-conscious. 
Therefore,  many  of  the  usual  problems  associated  with  group  dynamics  are  eliminated 
(Martino,  1978).  These  benefits  are  similar  to  those  offered  by  conqiuterized  group 
support  systems  (Laudon,  1999) 

Fewer  constraints. 

Many  times,  due  to  fiinding  or  scheduling  constraints,  the  members  of  a  group 
cannot  meet.  The  Delphi  technique  allows  individuals  to  interact  with  each  other  who 
cannot  be  brought  together  for  a  face-to-face  discussion  (Linstone,  1978). 

Statistical  Response. 
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It  allows  a  qualitative  measurement  to  be  made  of  an  inherently  qualitative 
response.  The  Delphi  attempts  to  measure  the  opinions  of  others  by  attaehing  a  numerieal 
result  to  the  response.  These  numbers  help  guide  the  study  and  show  when  consensus  has 
been  reached  (Martino,  1978). 

Multiple  Uses. 

The  Delphi  method  is  well  suited  for  multiple  applications.  It  has  been  used  in 
both  the  commercial  and  public  sector  for  numerous  reasons  including,  “exposing 
priorities  of  personal  values  and  social  goals,  explaining  the  pros  and  cons  associated  with 
potential  policy  options,  evaluating  budget  allocation,  examining  the  significance  of 
historical  events,  and  distinguishing  or  clarifying  perceived  and  real  human  motivations 
(Linstone,  1978). 

Weaknesses  of  Delphi 

Selection  of  experts. 

Each  member  selected  to  participate  in  the  Delphi  must  be  qualified  and  motivated. 
It  is  critical  to  the  process  that  the  responses  generated  eome  from  knowledgeable 
individuals.  These  individuals  must  also  be  committed  to  the  study  until  the  end.  Due  to 
the  iterative  nature  of  Delphi,  it  can  last  for  months,  therefore,  it  is  critical  that  aU 
individuals  are  available  for  the  entire  duration.  This  became  a  large  problem  during  this 
study  as  will  be  discussed  later. 

Discounting  the  future. 

Many  times  a  discoimt  rate  is  applied  to  distant  future  foreeasts.  This  occms  since 
individuals  perceive  potential  near-term  crisis  situations  as  more  significant  than  those 
several  years  away.  However,  this  rate  is  different  for  each  individual  and  therefore,  may 
skew  the  final  results  (Linstone,  1978).  However,  it  was  thought  that  this  would  not  be  a 
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problem  for  this  study  since  the  data  being  collected  revolved  around  near  future  or  even 
current  events  facing  the  experts.  Consequently,  there  was  less  potential  for  any  discount 
rate  to  be  applied  to  the  responses. 

Urge  to  simplify. 

When  considering  the  future,  people  tend  to  focus  on  a  small  number  of  potential 
innovations  and  overlay  them  onto  a  famihar  structure  context  from  the  past  or  present. 
This  leads  to  a  visualized  future  situation  that  does  not  consider  the  pervasive  mfluence  of 
change.  Therefore,  “intuitive  procedures  such  as  Delphi  depend  heavily  on  subjective 
probability  assessments”  (Linstone,  1978).  This  subjective  element  can  lead  to  experts 
confusing  desirability  and  familiarity  with  probability,  which  translates  into  them  selecting 
options  with  recognizable  elements  even  though  there  is  very  httle  evidence  to  support 
such  a  forecast  (Tversky,  1974). 

Illusory  expertise. 

This  problem  occurs  when  the  selected  experts  are  unable  to  escape  from  their 
current  thinking  paradigm.  Any  forecast  they  provide  might  only  be  a  view  of  incremental 
improvements  made  in  their  area  of  expertise  and  not  potentially  dramatic  or  fundamental 
improvements  in  the  overaU  process  (Linstone,  1978).  This  problem  can  be  intensified 
through  the  use  of  inbreeding,  which  occurs  when  a  panel  member  is  asked  to  suggest  the 
names  of  other  possible  participants.  For  this  study,  experts  were  selected  from  a  variety 
of  sources.  Therefore,  it  was  hoped  that  this  effect  was  minimized. 

Questionnaire  Construction 

Construction  problems  are  present  in  ahnost  any  data  collection  process  based  on 
questionnaires  (Dooley,  1995).  One  of  these  problems  is  format  bias,  which  is  the 
assumption  that  everyone  completing  the  survey  has  a  certain  cultural  background. 
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Therefore,  those  with  significantly  different  backgrounds  find  it  more  difficult  to  respond. 
However,  for  this  study  it  was  thought  that  due  to  the  homogeneity  of  the  selected  experts 
and  the  narrowly  defined  population,  this  bias  was  not  a  significant  factor. 

Item  construction  is  another  problem  that  can  be  introduced  when  questions  are 
unclear  or  badly  worded,  when  compound  items  are  used  or  when  the  options  of  a  closed- 
ended  question  become  confusing.  This  problem  was  addressed  by  having  research 
committee  members  review  and  comment  on  the  survey.  In  addition,  a  questionnaire 
examination  was  performed  to  help  ensure  that  all  questions  were  properly  worded,  easy 
to  understand,  and  had  only  one  possible  interpretation. 

Order  effects  can  also  taint  the  result  of  a  questionnaire.  These  are  introduced 
when  earlier  items  bias  the  responses  to  later  ones.  One  method  to  help  combat  this 
problem,  is  to  randomize  the  order  of  the  questions.  However,  “this  does  not  guarantee 
true  responses  to  the  items  in  their  different  orders”  (Dooley,  1995).  Therefore,  this  study 
applied  the  results  of  previous  research,  which  shows  it  is  best  to  start  with  more  general, 
less  threatening  questions  that  are  more  interesting  to  the  respondents  and  progress  slowly 
into  the  more  specific  areas.  This  ordering  was  based  on  the  author’s  subjective  opinion 
of  each  question.  Care  was  taken  to  ensure  that  questions,  which  tested  the  same  fector, 
were  not  located  next  to  each  other.  This  was  accomplished  by  codmg  each  question 
based  on  the  item  it  tested,  listing  the  questions  in  the  order  presented  and  verifying  that 
similarly  coded  questions  were  not  adjacent  to  each  other. 

Guiding  the  results. 

This  problem  is  encountered  when  the  responses  to  the  previous  round  are  altered 
by  the  administrators  in  the  hope  of  herding  the  next  round  in  a  desired  direction.  This 
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problem  was  miniinized  by  the  active  participation  of  committee  members  in  selecting  the 
questions  for  each  successive  round.  It  can  also  be  said  that  the  authors  had  no  stake  in 
the  results  of  the  study. 

Selection  of  Organziations 

For  this  study,  the  population  of  interest  was  the  Department  of  Defense, 
Department  of  the  Air  Force  and  Department  of  Defense  intelligence  community  agencies. 
For  the  purpose  of  this  paper  the  term  “agency”  will  be  used  as  a  global  reference  to  each 

entity. 

A  sampling  frame  was  generated  from  this  population  using  a  two-stage  approach. 
First,  participation  was  limited  to  agencies  within  40  miles  of  the  metropolitan  Washington 
D.C.  area.  This  population  reduction  helped  keep  the  data  collection  convenient  and 
minimized  travel  costs.  The  next  sample  was  drawn  by  personally  contacting  the  office  of 
the  CIO,  or  other  IT  outsourcing  decision  authority,  for  each  participating  agency.  A  bnef 
explanation  of  the  study  and  its  purpose  was  then  given  foUowed  by  a  request  to 
participate.  Since  the  concentration  of  Government  agencies  within  the  selected  locales  is 
so  high,  there  was  little  concern  that  the  desired  number  of  participating  agencies  would 
not  be  realized. 

As  with  other  Delphi  studies,  the  desired  number  of  participating  agencies  for  this 
study  was  seven  for  two  reasons  (Linstone,  1978).  First,  as  shown  in  Figure  8,  groups 
smaller  than  this  exhibit  quickly  decreasing  exponential  effects  on  average  group  error 
rate,  but  higher  values  show  only  a  shallow  monotonic  like  decrease. 
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NUMBEC  IN  GCOUP 

Figure  8.  Group  Size  vs.  Error  Rate  (Dalkey,  1969:11) 

Second,  as  seen  in  Figure  8,  any  group  of  seven  or  greater  has  a  sufficiently  high  reliability 
measure.  Reliability  refers  to  the  degree  to  which  observed  scores  are  free  from  errors  of 
measurement  (Dooley,  1999).  It  can  be  considered  how  consistent  the  scores  on  parallel 
forms  of  a  test  are  or  how  different  items  in  an  instrument  test  the  same  measure.  “For 
example,  two  sets  of  measmements  on  the  same  variable  for  the  same  individual  may  not 
have  exactly  the  same  value;  however,  repeated  measurements  of  a  series  of  individuals 
will  show  some  consistency”  (Courtois,  1994,37).  Measurement  theory  suggests  that  the 
higher  number  of  items,  the  more  reliable  the  measure  (Dooley,  1999).  However,  this  is 
true  only  if  the  items  have  consistency  and  high  inter-item  reliability.  For  the  purpose  of  a 
Delphi  study,  reliability  is  the  likelihood  that  two  different  groups,  comprised  of  similarly 
qualified  participants,  will  arrive  at  the  same  conclusion.  Figure  9,  was  developed  by 
randomly  selecting  pairs  of  variously  sized  groups  and  “correlating  the  median  responses 
of  the  pairs  on  twenty  questions”  (Dalkey,  1969). 
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RELIABILITY  VS  GROUP  SIZE 


Figure  9.  Reliability  vs.  Group  Size  (Dalkey,  1969:13) 

Selection  of  Experts 

The  selection  of  the  experts  in  a  Delphi  study  is  critical  and  varies  among 
application.  It  is  especially  important  \vhen  considering  IT  outsourcing  issues  since 
research  has  shown  that  different  groups  within  an  organization  can  view  IT  outsourcing 
differently  (Gefen,  1998).  As  listed  above,  these  experts  were  selected  using  a 
stratification  approach.  At  the  beginnmg  of  the  study,  ten  members  had  agreed  to 
participate.  Table  2  below  lists  them  along  with  each  round  they  contributed  to. 
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Table  2.  Study  Participants 


Position 

Agency 

Expert  # 

Participated  in: 

Round  #1 

Round  #2 

Round  #3 

CIO 

IC  agency  #1 

3 

X 

CIO 

IC  agency  #2 

4 

X 

X 

X 

CIO 

Federally  Funded 
Research  and 
Development  Center 
(FFRDC) 

9 

X 

X 

X 

Deputy  CIO 

IC  agency  #3 

7 

X 

X 

X 

Deputy  CIO 

IC  agency  #4 

1 

X 

Deputy  CIO 

IC  agency  #5 

2 

X  ^ 

X  n 

X 

Deputy  CIO 

Office  of  the  Secretary 
of  the  Air  Force 

5 

X 

CIO  Advisor 

FFRDC  /  IC  agency  #1 

10 

X 

X 

X 

Deputy 
director  of 
Communicati 
ons 

Acquisition 

IC  agency#! 

8 

X 

X 

X 

Operations 

Director 

Office  of  the  Secretary 
of  Defense 

6 

X 

X 

X 

As  can  be  seen,  each  participant  was  assigned  a  number.  This  helped  keep  the  responses 
confidential.  To  re-enforce  the  strategic  flavor  of  this  study,  the  two  FFRDC  participants 
were  at  the  chief  scientist  and  vice  president  level,  two  participants  were  general 
management  (GM)-15’s,  one  was  a  an  Air  Force  Lieutenant  Colonel,  two  were  senior 
executive  service/senior  intelligence  service  (SES/SIS)-3’s,  two  were  SES/SIS-4’s  and 
one  was  an  SES/SIS-5. 

It  is  conceivable  that  by  agreeing  to  participate  in  this  study  a  samplii^  bias  could 
have  been  introduced.  This  would  occur  if  one  potential  expert  accepted  or  declined  to 
participate  for  reasons  not  applicable  to  other  members.  Examples  of  these  reasons 
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include  over  tasking,  prior  comniitments,  and  disinterest  in  the  study.  However,  it  was 
believed  that  these  reasons  are  the  same  for  each  agency  and  therefore,  uniform 
throughout  the  sampling  frame.  Consequently,  the  chances  of  an  agency  not  participating 
can  be  considered  random. 

The  optimum  mix  of  experts  included  the  proper  number  of  stakeholders,  experts  and 
facilitators.  Stakeholders  are  those  who  are  directly  affected  by  the  decisions,  experts  are 
those  who  have  an  applicable  specialty  or  relevant  experience,  and  facilitators  are  those 
who  have  skills  in  clarifying,  organizing,  and  synthesizing.  While  there  are  no  heuristics 
for  determining  the  proper  ratio  of  these  individuals  in  a  Delphi  group,  each  application 
can  help  drive  the  proper  balance.  For  this  study  the  “options  and  interests  are  clear  but 
acceptance  of  direction  and  action  is  fractionated.”  (Scheele,  1975,68).  Research  has 
shown  that  in  this  situation  it  is  not  inappropriate  for  stakeholders  to  dominate  a  group 
(Scheele,  1975).  For  this  research,  stakeholders  are  usually  the  CIO  of  an  agency. 
However,  since  expert  and  fecilitator  opimon  was  still  welcomed,  if  a  CIO  wished  to  send 
a  technical  representative  in  their  stead,  it  was  not  declined. 

Development  of  the  Data  Gathering  Instrument 

Each  instrument  was  a  questionnaire  developed  to  determine  (in  order  of  priority) 

1)  if  the  proposed  theory  was  accurate;  2)  if  it  was  not,  where;  3)  offer  a  refined  version  of 
the  theory.  Each  questionnaire  had  two  portions.  The  first  was  a  set  of  closed-ended 
descriptive  and  prescriptive  questions  that  satisfied  the  first  and  second  objectives.  The 
descriptive  questions  used  a  five-point  Likert  scale  ranging  from  strongly  disagree  to 
strongly  agree  for  the  question;  amount  of  consideration  given.  The  prescriptive 
questions  used  the  same  scale  for  the  question;  level  of  agreement.  All  the  questions  were 
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structured  to  mirror  the  proposed  relationship  that  exists  between  the  determinants  they 
were  testing  for  and  the  dependent  variable;  amount  of  outsourcing 

The  second  portion  helped  satisfy  the  third  objective  of  the  questionnaire  by 
providing  open-ended  questions  that  allowed  the  experts  to  propose  different  ideas  or 
explain  any  anomalous  responses. 

Round  One  Questionnaire 

The  first  survey  helped  form  a  baseline  consensus  of  the  working  theory.  As 
shown  in  Appendbc  A,  the  closed-ended  questions  helped  to  quantitatively  establish  the 
acceptance  of  the  theory,  while  the  open-ended  questions  helped  guide  the  development  of 
new  questions  for  the  second  roimd.  If  for  instance,  a  determinant  was  omitted  in  the  first 
round  instrument,  but  listed  as  an  important  factor  by  one  of  the  experts,  it  was  included  in 
the  next  round. 

Results  and  Formulation  of  the  Second  Questionnaire 

The  second  survey  was  based  on  the  responses  obtained  fi'om  the  first.  This 

progression,  as  defined  by  Delphi  procedures,  helped  fiirther  define  what  the  strengths  and 

weakness  of  the  theory  were  and  proposed  new  ideas  based  on  the  responses  to  the  open- 

ended  questions. 

The  second  questionnaire  was  exactly  the  same  as  the  first  except  for  the  inclusion  of 
new  fectors  suggested  by  the  participants  and  the  removal  of  the  lower  scoring  items. 
Determining  which  factors  had  the  lowest  score  was  done  by  averaging  the  answers  for 
each  item  and  establishing  a  threshold  value.  In  this  case,  the  value  selected  was  3.00. 
While  the  selection  of  this  number  was  somewhat  arbitrary,  it  is  consistent  with  pervious 
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studies  (ScheflQer,  1999).  Additionally,  it  allowed  for  a  convenient  reduction  of 
approximately  50%  in  the  total  number  of  questions. 

It  should  be  noted  that  all  calculations  were  performed  at  an  item  level.  Therefore,  the 
removal  of  any  questions  was  conducted  independent  of  the  fectors  they  were  associated 
with.  Consequently,  it  was  possible  for  one  item  from  a  factor  to  remain  in  subsequent 
rounds,  even  though  aU  other  questions  associated  with  that  category  might  have  been 
deleted.  Of  course  if  this  occurred,  it  might  suggest  that  such  factors  were  less  important 
than  factors  that  retained  most  of  their  items  through  the  rounds.  This  process  was  used 
to  prevent  the  deletion  of  an  important  category  or  item  simply  because  they  may  have 
been  incorrectly  associated  with  each  other. 

Round  Two  Questionnaire 

During  the  second  rovmd,  each  participant  was  given  statistical  feedback  based  on 
the  previous  survey.  As  shown  in  Appendix  B,  the  mean  of  each  remaining  closed-ended 
question  was  provided  as  well  as  the  respondents  previous  score. 

Also  each  member  that  provided  a  response  greater  than  1 .25  standard  deviation 
away  from  the  group  average  was  asked  to  comment  on  it.  It  was  hoped  that  this  would 
help  determine  if  these  answers  were  based  on  information  the  entire  group  should  be 
aware  of,  or  if  it  was  an  issue  applicable  only  to  a  specific  agency.  Any  information  this 
process  uncovered  that  was  deemed  appropriate  to  bring  to  the  attention  of  the  entire 
group,  was  included  in  the  third  questionnaire 

Round  Three  Questionnaire 

Research  and  empiricism  has  show  that  after  three  iterations  any  additional 
consensus  obtained  within  an  expert  group  will  be  negligible  (Dickson,  1984).  Therefore, 
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it  was  expected  that  this  round  would  capture  the  final  analytical  theory.  However,  since 
additional  consensus  is  negligible  after  three  rounds,  it  is  difficult  to  determine  a  priori 
when  consensus  be  reached. 

For  this  reason,  some  have  proposed  that  consensus  in  a  Delphi  should  not 
necessarily  be  measured  by  the  percentage  of  votes  that  fall  within  a  prescribed  range,  but 
rather  the  stability  of  the  respondents’  vote  distribution  curve  over  successive  rounds 
(Scheibe,  1975).  A  marginal  change  of  less  than  15  percent  offers  a  useful  definition  of 
stability.  It  has  been  noted  that  if  a  round  provides  less  than  a  1 5  percent  change  from  the 
preceding  round,  the  exercise  should  be  stopped  (Scheibe,  1975). 

For  this  study,  consensus  will  be  measured  by  the  standard  deviation  and 
interquartile  range  (IQR)  of  the  responses.  These  statistics  will  be  plotted  to  show  the 
trend  towards  perfect  consensus  (standard  deviation  =  0) 

Testing  of  the  Data  collection  Instrument 
Faculty  Review 

Each  questionnaire  was  assessed  by  the  academic  advisor  of  this  research  to 
determine  if  the  item’s  were  clear  and  focused  and  whether  the  instrument  possessed 
content  validity.  This  individual  was  familiar  with  the  domain  and  able  to  help  edit  the 
questions  to  help  avoid  misinterpretations  by  the  respondents. 

Questionnaire  Examination 

To  help  ensure  that  the  instrument  would  function  as  intended,  it  was  the  subject 
of  a  comprehensive  review.  The  main  purpose  of  this  assessment  was  to  determine  if  the 
instrument  exhibited  proper  item  construction.  The  questions  and  instructions  were 
inspected  for  typographical  errors,  readability,  multiple  interpretations,  order  effect,  and 
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whether  they  were  understandable.  The  participants  for  this  review  included  members  of 
the  March  2001  graduate  information  management  (GIR)/graduate  information  systems 
(GIS)  class  at  the  Air  Force  Institute  of  Technology  (AFIT). 

The  review  used  a  two-phased  approach.  During  the  first  phase,  two  students 
were  asked  to  examine  the  instrument.  Once  their  comments  were  considered  and 
included,  the  procedure  was  repeated  with  two  different  students. 

Distribution  of  Data  Gathering  Instrument 

To  help  minimize  inadvertent  transfer  of  information  outside  of  the  expert  group 

(Dalkey,  1969)  the  first  questionnaire  was  hand  delivered  to  each  participant  along  with  a 
cover  letter  and  specific  instructions  on  how  to  complete  it.  These  items  are  included  in 
Appendix  A.  The  last  two  surveys  were  delivered  via  electronic  facsimile  transmission 
(fex)  or  electronic  message  (e-mail)  with  updated  cover  letters  and  instructions.  These  are 
shown  in  Appendix  B  and  C  respectively.  It  was  requested  that  all  completed 
questionnaires  either  be  fexed  to  the  study’s  dedicated  fax  number,  mailed,  or  e-mailed 
back. 

Data  Processing 

Data  processing  was  performed  using  Microsoft  Excel.  Immediately  after  a  survey 
was  returned,  the  results  were  entered  into  an  Excel  spreadsheet.  One  main  area  of 
analysis  was  the  summation  of  the  Likert  data  into  mean,  standard  deviation  and  IQR  data. 
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IV.  Data  Descriptions  and  Analysis 


Introduction 

The  objective  of  this  chapter  is  to  provide  the  results  of  the  Delphi  study.  An 
overview  of  each  round  will  be  presented  along  with  examples  of  the  data.  A  complete 
listing  of  all  the  raw  data  for  each  of  the  three  rounds  can  be  found  in  Appendix  D,  E,  and 
F. 

Round  One 

The  first  objective  for  the  round  one  was  to  roughly  determine  which  of  the 
proposed  determinants  fi-om  the  literature  search  were  most  applicable.  The  second  was 
to  use  open-ended  questions  to  obtain  any  additional  items  fi^om  the  experts,  not  specified 
in  the  literature.  Typically,  in  the  first  round  of  a  Delphi  experiment  the  participants  will 
list  items  that  they  feel  are  most  important  to  the  research  questions.  However,  since 
round  one  started  with  the  list  generated  by  the  literature  search,  open-ended  questions 
were  included  in  the  survey  to  allow  the  respondents  to  include  any  other  items  they 
thought  were  important.  The  scale  for  this,  and  all  subsequent  rounds,  was  a  five-point 
Likert  scale. 

Results 

After  a  participant  submitted  a  completed  questionnaire,  the  responses  were 
entered  into  a  Microsoft  Excel  spreadsheet.  An  average  and  standard  deviation  were  then 
calculated  for  each  item.  Table  3  below  shows  an  example  of  the  spreadsheet  used  for 
this.  Here,  the  first  four  responses  fi-om  experts  #1  and  #2  are  displayed  along  with  the 
resulting  average  and  standard  deviation  fi)r  those  items.  The  naming  convention  used  to 
identify  study  questions  in  the  surveys,  is  table  number  followed  by  question  number.  As 
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an  example,  Tl-01  is  table  one,  question  one  and  T2-01  is  table  two,  question  one. 
However,  since  there  are  a  different  number  of  questions  in  each  round,  most  items  will 
have  a  different  number  assigned  to  them  for  each.  Therefore,  for  ease  of  reading,  all 
questions  have  been  re-coded  to  a  universal  numbering  scheme  for  presentation  in  the  text. 
However,  it  should  be  noted  that  the  survey’s  themselves  will  retain  their  original  number 
convention.  For  clarification  purposes,  Appendbc  G  contains  an  inter-round  question 
numbering  map  that  matches  any  question  in  the  surveys  to  its  universal  code. 


Table  3.  Example  of  Raw  Data 


Question  # 

Expert  #1 

Expert  #2 

... 

Avg 

Std  Dev 

1 

4 

3 

3.8 

1.135 

2 

5 

2 

3.8 

1.398 

3 

5 

5 

4.1 

1.197 

4 

4 

3 

3.667 

0.866 

•  •• 

... 

... 

... 

Following  the  mean  and  standard  deviation  calculations,  the  data  were  reordered 
based  on  the  sub-factors  to  make  the  results  easier  to  work  with.  To  accomplish  this,  each 
sub-factor  was  assigned  an  alphanumeric  designator  based  on  its  associated  factor.  For 
example,  the  first  cost  sub-factor,  cost  reduction,  was  given  the  designator  Cl,  the 
second,  improved  cost  controls,  was  given  C2,  and  so  on.  The  data  were  then  sorted 
based  on  these  designators.  An  example  for  expert  #1  and  #2  is  listed  in  Table  4. 
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Table  4.  Example  of  Data  Sorted  Based  on  Item  Type 


Question  # 

Item  Type 

Expert  #1 

Expert  #2 

... 

Avg 

Std  Dev 

29 

T6 

3 

1 

2.111 

0.928 

44 

T6 

4 

1 

2.889 

1.364 

28 

T7 

2 

4 

3.100 

1.197 

35 

TV 

4 

3 

3.889 

1.167 

•  •• 

... 

... 

... 

... 

During  the  analysis  it  was  determined  that  questions  associated  with  the  business 
factors  inability  to  write  an  adequate  service  level  agreement  and  inability  to  manage  a 
contract  had  been  incorrectly  phrased.  Both  of  these  fectors  are  a  risk.  However,  the 
questions  were  worded  as  if  they  were  benefits.  The  average  responses  to  these  questions 
were  high,  consequently  they  should  have  been  retained.  Yet,  when  it  is  considered  that 
they  should  have  been  phrased  to  elicit  the  opposite  response,  an  argument  is  formed  that 
the  responses  should  have  been  low,  which  would  have  caused  their  average  scores  to  fall 
below  the  threshold  value  and  result  in  the  item  being  removed  from  futine  rounds.  To  be 
conservative,  all  questions  associated  with  these  two  factors  were  re-worded  in  a  negative 
manner  to  reflect  the  risk  factor  they  were  testing  for.  These  questions  were  then  included 
in  the  second  questionnaire. 

A  complete  listing  of  the  remaining  data  ranked  by  mean  is  presented  in  Table  5. 
Figure  10  is  a  plot  of  each  of  the  mean  scores  based  on  the  rank  of  the  question.  As 
shown  by  the  r-squared  value  of  0.97,  the  continuum  of  these  values  is  very  linear. 
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Table  5.  First  Round  Data  Sorted  Based  on  Mean 


3.222  F5  Increased  transactional  costs  (contract  processing,  administration,  coordination,  etc.) _ 

3.200  B3  Act  as  a  change  agent  for  reorganization _ ^ ^ - 

3.200  E2  I  believe  some  IT  managers  are  interested  in  IT  outsourcing  only  because  other  IT  managers  in  the  government  are 

3.200  F4  Protect  and  stabilize  IT  budgets  (e.g.  placing  IT  on  contract  to  protect  funding) _ 


Avg  Item  Type  Questions _ _ _ 

3,200  T5 _ Complications  between  vendor  and  customer  technical  IT  interfaces _ 

3.125  F5  ~  Potential  for  hidden  costs  (e.g.  unexpected  costs  for  assumed  IT  services) 

3.125  _ T2 _ Retain  competent  IT  professionals _ _ _ 

3.111  T4  Feelings  that  vendor  will  not  be  able  to  solve  agency  specific  IT  requirements 


2.600  B1 _ Ability  of  IT  outsourcing  to  flatten  organizations _ _ _ 

2.600  E3  I  believe  concern  over  displaced  IT  staff  members  affects  my  decision  to  outsource  _ 

2.600  FI  Lower  costs  due  to  a  vendor’s  tighter  control  over  fringe  benefits _ _ 

2.600  P3  I  believe  some  IT  managers  consider  outsourcing  to  expose  exaggerated  claims  made  by  vendors,  thereby  workini 


Formulation  of  Second  Questionnaire 

Before  beginning  the  process  of  determining  which  questions  would  be  retained  for 
the  second  questionnaire,  each  response  was  first  tested  to  determine  if  it  fell  1.25 
standard  deviations  firom  the  mean.  These  results  would  later  be  combined  with  the  final 
set  of  retained  questions  so  a  respondent  would  have  an  opportunity  to  comment  on  why 
they  felt  their  response  might  have  been  exceptionally  high  or  low.  This  number  was 
selected  since  it  was  the  lowest  value  that  still  allowed  each  participant  at  least  one 
opportunity  to  provide  feedback. 

Following  this,  a  threshold  value  of  3.00  was  appUed  to  each  items  average.  All 
averages  above  the  threshold  were  retained  for  the  next  round,  while  all  those  below  were 
deleted.  As  mentioned  earlier,  while  the  selection  of  this  number  was  somewhat  arbitrary, 
it  is  consistent  with  pervious  studies  (Scheffler,  1999).  Additionally,  it  allowed  for  a 
convenient  reduction  of  approximately  50%  in  the  total  number  of  questions,  retaining  43 
out  of  the  original  83.  The  retained  questions  were  then  automatically  combined  along 
with  each  participants  previous  response  and  whether  it  was  above  or  below  the  defined 
number  of  deviations.  Table  6,  lists  the  first  7  questions  retained  for  the  second  survey 
along  with  their  alphanumeric  designators  (Type),  averages  (Avg),  standard  deviations 
(Std  D),  boundaries  for  selected  minimum  and  maximum  standard  deviations  (1.25), 
respondent  #1,  roxmd  one  answers  (1-Ans),  indicator  of  whether  respondent  #rs  answers 
were  above  or  below  the  preset  minimum  and  maximum  standard  deviation  value  of  1.25 
(1-Out),  respondent  #2,  round  one  answers  (2-Ans),  and  finally  an  indicator  of  whether 
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respondent  #2’s  answers  were  above  or  below  the  preset  minimum  and  maximum  standard 

deviation  value  of  1.25  (2-Out). 

Table  6.  Sub-set  of  Questions  Selected  for  Second  Survey 


Question  # 

Type 

Avg 

StdD 

1.25 

1.25 

1-Ans 

1-Out 

2-Ans 

2-Out 

... 

01 

FI 

3.800 

1.135 

5.219 

2.381 

4 

3 

11 

FI 

3.800 

1.229 

5.337 

2.263 

3 

4 

14 

F2 

4.000 

0.471 

4.589 

3.411 

4 

3 

Low 

39 

F2 

3.000 

0.866 

4.083 

1.917 

3 

3 

18 

F3 

3.100 

0.876 

4.194 

2.006 

3  ^ 

2 

Low 

12  ^ 

F4 

3.400  ^ 

1.075 

4.744 

2.056 

3 

3 

33 

F4 

3.200 

1.135”^ 

4.619 

1.781 

2 

4 

... 

... 

... 

... 

... 

... 

... 

... 

... 

To  ensure  that  the  spreadsheet  was  calculating  the  correct  values,  over  20%  of  the 
final  43  questions  were  selected  at  random  and  verified  by  hand  calculations. 

Informal  feedback  fi'om  the  first  questionnaire  showed  that  some  participants 
thought  the  flow  of  the  instrument  was  rough.  They  thought  it  did  not  read  well  and  was 
therefore  difficult  to  follow  and  respond  to.  This  was  anticipated  due  to  the  tight 
adherence  to  guidelines  used  to  minimize  order  effect.  However,  for  the  second  survey  it 
was  thought  that  these  guidelines  could  be  relaxed.  Consequently,  the  questions  were 
reordered  to  present  those  testing  for  benefits  first  followed  by  those  testing  for  risks. 
These  categories  were  fiorther  ordered  based  on  the  presentation  of  the  theory  in  chapter 
two.  It  was  hoped  that  this  would  produce  a  smoother  flowing  instrument. 

As  previously  mentioned,  open  ended  questions  were  included  in  the  survey  to 
allow  each  participant  to  add  additional  factors  they  thought  were  important  to  an 
outsourcing  decision.  When  an  individual  provided  a  response  to  one  of  these  questions, 
it  was  entered  into  a  separate  file  and  if  necessary,  edited  so  that  it  was  easier  to 
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understand.  Each  new  question  was  then  categorized  as  either  having  a  positive  or 
negative  relationship  on  the  decision  to  outsourcing.  As  previously  mentioned,  those  with 
a  positive  relationship  were  labeled  as  a  benefit  and  those  with  a  negative  relationship 
were  labeled  as  a  risk.  This  allowed  the  questions  to  be  ordered  using  the  new  process 
listed  above. 

Following  this,  an  attempt  was  made  to  match  each  of  the  new  items  to  an  already 
theorized  sub-factor.  This  was  done  to  more  easily  track  the  questions  through  future 
rounds.  If  the  question  was  able  to  matched,  the  appropriate  alphanumeric  designator  was 
simply  assigned  to  the  new  item.  If  a  new  question  could  not  be  easily  mapped  to  an 
existing  factor,  a  new  factor  was  created.  An  N  precedes  the  designator  for  any  new  sub¬ 
factor.  This  process  was  then  reviewed  by  advisors  to  ensure  that  it  any  mis- 
categorizations  were  minimized.  Each  new  item  was  then  assigned  a  number  based  on  the 
universal  code  previously  described.  For  these  new  items  the  numbering  was  simply 
continued  from  the  last  item.  However,  to  denote  that  these  were  new  questions,  an  N 
precedes  each  of  their  assigned  numbers.  Table  7  presents  a  complete  listing  of  the  14 
new  items  added  to  the  second  questionnaire  along  with  their  assigned  alphanumeric 
designators  and  numbers.  The  only  item  that  could  not  be  assigned  to  an  already 
theorized  factor  was  a  suggestion  regarding  best  industry  practices  and  improved  process. 
For  this  item  a  category  labeled  as  additional  was  developed.  Since  this  was  a  new 
category,  this  item  was  given  the  designator  NAl  as  shown. 
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Table  7.  New  Questions 


Question 

# 

New  question 

Item 

Type 

Table  #1; _ _ _ _ _ _ — - - 

N84 

Congressional  Interest  _ _ — - - 

B2 

N85 

Create  aaencv  wide  business  practice 

F2 

N86 

Ability  of  IT  outsourcing  to  make  interoperability  between  systems  LANs 
better  - - - 

T3 

N87 

Duality  of  Intra-aaencv  relationships  (between  IT  and  business  unitsj 

NEl 

N88 

Lack  of  a  clearly  delineated  set  of  systems,  costs  and  service  levels 

B4 

N89 

Controlling  different  security  levels  of  data  without  security  mcident 

_ ^ _ 

Table  #2:  - 

N90 

I  believe  that  many  agencies  feel  that  outsourcing  is  either  mandated  or  so 
strongly  encouraged  as  to  be,  in  essence  mandated 

B2 

N91 

I  believe  that  many  senior  IT  managers  see  outsourcmg  as  an  end  m  itself 
rather  than  a  strategy  to  achieve  a  goal - 

B2 

N92 

I  believe  outsourcing  activities  are  not  initiated  by  IT  organizations.  External 
actors  loom  large  in  decisions _ _ _ _ _ _ _ 

NE2 

N93 

I  believe  some  IT  managers  view  outsourcing  as  a  means  to  adopt  industry 
best  practices  and  improve  processes 

NAl 

N94 

I  believe  some  IT  managers  view  outsourcing  as  a  means  to  mitigate  the 
inability  of  government  to  remove  incompetent  IT  workers 

B4 

N95 

I  believe  the  cost  of  outsourcing  is  not  clearly  understood 

F5 

N96 

I  believe  outsourcing  should  not  be  done  without  a  comprehensive  business 
assessment  to  establish  value  _ _ _ 

F5 

N97 

I  believe  managers  do  not  determine  best  value  prior  to  outsourcmg 

F5 

A  complete  list  of  all  fectors,  sub-fectors,  and  associated  alphanumeric  designators  are 
presented  in  Appendix  H. 


Round  Two 

The  second  round  had  five  objectives.  The  first  was  to  retest  the  subset  of 
questions  developed  fi-om  round  one  in  the  hopes  of  gaining  consensus.  The  second  was 
to  retest  the  business  related  items  that  were  reworded.  The  third  was  to  test  any  new 
questions  developed  as  a  result  of  the  responses  to  the  open  ended  questions  m  the  first 
round.  The  fourth  objective  was  to  obtain  comments  fi-om  the  experts  if  their  previous 
response  fell  more  than  1.25  standard  deviations  fi-om  the  mean  and  the  fifth  was  to  obtain 
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any  new  items  from  open-ended  questions.  The  scale  used  on  this  survey  was  the  same 
five  point  Likert  Scale  used  in  the  first  round. 

Results 

The  raw  data  collected  from  this  questionnaire  was  entered  and  reduced  in  nearly  the 
same  manner  as  the  first  survey.  The  only  difference  was  that  the  data  did  not  need  to  be 
reordered,  since  it  was  already  grouped  based  on  sub-factors. 

Also,  as  anticipated,  the  newly  worded  version  of  the  questions  relating  to  the  inability 
to  write  an  adequate  service  level  agreement  and  inability  to  manage  a  contract  almost 
all  had  an  average  value  substantially  lower  than  even  the  first  survey’s  threshold  of  3.0. 
Yet,  it  should  be  noted  that  three  of  these  items  did  score  at  or  above  the  threshold. 
However,  these  items  did  not  directly  test  the  relationship  between  the  proposed  sub¬ 
factors  and  the  dependent  variable,  but  rather  simply  tested  for  the  existence  of  the  sub¬ 
factor  as  it  relates  to  the  dependent  variable.  While  this  may  prove  useful  for  fiiture 
research,  it  was  decided  that  since  the  proposed  relationship  between  these  sub-factors 
and  the  dependent  variable  had  already  been  rejected,  no  further  investigation  in  this  area 

was  warranted. 

A  listing  of  the  57  questions  asked  in  round  two  are  presented  in  a  rank  order  format 
in  Table  8.  This  table  includes  the  43  question  retained  from  the  first  round  along  with  the 
14  new  items  suggested  by  the  experts.  Table  9  displays  the  same  information  for  the  re¬ 
worded  questions  described  above. 
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Table  8.  Original  Second  Round  Data  Sorted  Based  on  Mean 


Questions 

Improve  the  long-term  financial  predictability  of  IT  budgets 

Loss  of  critical  IT  skills  base 

Security  concerns  over  allowing  outside  security  specialists  access  to  IT  systems 

Congressional  Interest 

Ability  of  IT  outsourcing  to  make  interoperability  between  systems  LANs  better 

I  believe  that  many  agencies  feel  that  outsourcing  is  either  mandated  or  so  strongly  encouraged  as  to  be,  in  essence 
mandated 

Possibility  that  vendor  will  disclose  classified  data 

Security  concerns  about  subcontractors 

Inability  of  a  vendor  to  guarantee  interoperability  with  system  they  don't  control 

Quality  of  Intra-agency  relationships  (between  IT  and  business  units) 

I  believe  some  IT  managers  are  interested  in  IT  outsourcing  only  because  other  IT  managers  in  the  government  are 

Retain  competent  IT  professionals 

Becoming  overly  dependent  on  contractors 

Feelings  that  vendor  will  not  be  able  to  solve  agency  specific  IT  requirements 

Record  all  costs  incurred  by  the  users  . 

Protect  and  stabilize  IT  budgets  (e.g.  placing  IT  on  contract  to  protect  funding) 

Act  as  a  change  agent  for  reorganization 

Potential  for  hidden  costs  (e.g.  unexpected  costs  for  assumed  IT  services) 

Increased  transactional  costs  (contract  processing,  administration,  coordination,  etc.) 

Loss  of  control  over  timing  and  quality  of  outputs 

Concerns  over  a  vendors  e-mail  security  procedures  when  discussing  agency  issues 

I  believe  that  many  senior  IT  managers  see  outsourcing  as  an  end  in  itself  rather  than  a  strategy  to  achieve  a  goal 

Convert  large  IT  capital  budgets  into  more  flexible  operating  budgets 

Access  to  a  vendor’s  large  research  and  development  efforts 

Lose  control  over  IT  services _ _ _ _ _ _ _ — 

rnmnUpatintiQ  vendor  and  customer  technical  IT  interfaces 

Concerns  over  how  a  vendor  disposes  agency  information  after  it  is  no  longer  needed 

I  believe  the  length  of  time  an  IT  manager  knows  they  will  be  in  a  position  affects  their  decision  to  outsource 
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Table  9.  Re-worded  Second  Round  Data  Sorted  Based  on  Mean 


Formulation  of  Third  Questionnaire 

The  responses  were  again  tested  to  determine  if  responses  fell  outside  a  defined 
number  of  standard  deviations.  The  selected  number  of  standard  deviations  was  again 
1 .250.  When  this  was  performed,  a  much  smaller  number  of  them  fell  outside  the  range  as 
compared  to  the  first  round  and  many  of  those  that  did  were  for  the  same  questions  that 
were  previously  out  of  the  range.  Therefore,  for  these  reasons,  it  was  decided  that  in  the 
next  rovmd  the  respondent  would  not  be  asked  to  comment  on  why  their  response  may 
have  been  exceptionally  high  or  low. 

Once  the  means,  standard  deviations,  and  IQRs  were  calculated,  a  threshold  of 
3.714  was  applied.  This  forced  a  reduction  in  the  number  of  questions  firom  43  to  23. 
When  this  was  performed,  eight  of  the  14  new  questions  produced  fi-om  responses  to  the 

first  roimds  open-ended  questions  remained. 

To  determine  if  consensus  was  being  achieved,  the  change  in  standard  deviations 
for  the  remaining  questions  that  were  asked  in  both  the  first  and  second  rounds  were 
plotted  as  shown  in  Figure  11.  In  this  plot,  consensus  would  display  itself  as  a  decrease  in 
the  standard  deviations  fiar  each  item  with  perfect  consensus  being  represented  by  a 
standard  deviation  of  zero  (Dickson,  1984). 
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Figure  11.  Standard  Deviations  for  Rounds  1-2 


It  can  be  seen,  however,  that  the  standard  deviation  for  10  of  the  15  questions 
went  up.  These  results  were  even  more  confounding  when  the  interquatile  range  (IQR) 
was  plotted  for  the  same  items.  IQR  is  the  difference  between  the  third  quartile  rank  score 
and  the  first  quartile  rank  score  and  can  be  used  as  another  measure  of  consensus 
(Dickson,  1984).  Figure  12  shows  that  the  IQR  for  many  items  went  down  even  when 
their  corresponding  standard  deviations  went  up. 
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Figure  12.  Interquartile  Range  for  Rounds  1-2 


In  an  attempt  to  explain  this  phenomenon,  the  stabiUty  of  each  item  was  analyzed. 
This  measurement  attempts  to  maximize  the  utility  of  the  voting  resxilts  by  determining  the 
constancy  of  each  individual’s  responses  over  successive  rounds  (Scheibe,  1975).  If 
stability  exists  without  consensus  being  formed,  it  might  be  an  indicator  that  unconsensual 
distributions  or  other  anomalies  exist  that  would  preclude  consensus  from  being  formed 
further.  To  calculate  the  stability,  the  histograms  were  subtracted  columnwise  and  the 
absolute  value  of  the  result  taken.  Table  10  shows  this  calculation  for  item  number  1  and 

7. 
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Table  10.  Example  of  Stability  Measurement  Calculation  in  Round  Two 


Item  #1 


Questionnaire  #1  response 
Questionnaire  #2  response 

Histograms 
Questionnaire  #1 
Questionnaire  #2 

Absolute  difference  between  rounds  1-2 

Rating 

Total  units  of  change 
Net  person-changes 
Number  of  participants 
Percent  change 

Item  #7 


Questionnaire  #1  response 
Questionnaire  #2  response 

Histograms 

Questionnaire  #1 
Questionnaire  #2 

Absolute  difference  between  rounds  1-2 

Rating 

Total  units  of  change 
Net  person-changes 
Number  of  participants 
Percent  change 


Expert  Response 

1 

2 

3 

4 

5 

6 

7 

3 

4 

4 

5 

4 

5 

1 

3 

4 

4 

4 

4 

5 

1 

Bins 

1 

2 

3 

4 

5 

r 

0 

T 

3 

2 

1 

0 

1 

4 

1 

0 

0 

0 

1 

1 

2 

1 

7 

14.29% 


Expert  Response 

1 

2 

3 

4 

5 

6 

3 

4 

5 

4 

4 

5 

2 

5 

5 

4 

5 

5 

Bins 

1 

2 

3 

4 

5 

0 

1 

1 

3 

2 

0 

2 

0 

1 

4 

0 

1 

1 

2 

2 

6 

3 

7 

42.86% 


As  shown,  the  absolute  values  of  the  difference  between  histograms  are  added  to 
determine  the  total  units  of  change.  Since  any  one  participant’s  change  of  opinion  is 
reflected  in  the  histogram  difference  by  two  timts  of  change,  net  person-changes  are 
computed  by  dividing  total  units  of  change  by  two.  Finally,  the  percentage  change  is 
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determined  by  dividing  net  changes  by  the  nximber  of  participants  (Scheibe,  1975). 
Typically,  a  change  of  less  than  15%  represents  a  state  of  equilibrium.  Any  two 
distribution  that  show  marginal  changes  of  less  than  15%  may  be  said  to  have  reached 
stability”  (Sheibe,  1975;  278).  With  only  7  respondents,  this  becomes  a  very  rigorous  test 
since  only  items  with  net-person  changes  of  one  or  below  will  pass. 

The  results  of  these  calculations  for  each  of  the  15  questions  asked  in  the  first  two 
rovmds,  and  retained  for  the  third,  are  present  below  in  Table  11.  This  table  also  annotates 
if  the  standard  deviation  of  the  question  went  up  or  down  between  the  rounds. 


Table  11.  Stability  Measurements  for  Questions  Retained  for  Survey  Three 


Question 

Stability 

Std  Dev  tup  or  down) 

1 

14.29% 

_ UE _ 

3 

14.29% 

_ ue _ 

7 

42.86% 

Down 

11 

28.57% 

_ ue _ 

14 

14.29% 

_ UE _ 

19 

14.29% 

UE _ 

23 

0.00% 

UE _ 

26 

14.29% 

UE _ 

40  n 

14.29% 

Up _ 

43 

14.29% 

Up _ 

66 

28.57% 

Down 

73 

28.57% 

Down 

77 

28.57% 

Down 

80 

14.29% 

Down 

81 

14.29% 

Up _ 

As  shown,  all  but  one  of  the  questions  whose  standard  deviation  went  up  (1 1)  exhibited  a 
stability  less  than  15%.  Additionally,  four  out  of  the  five  questions  whose  standard 
deviation  went  down  had  a  stability  of  greater  than  15%.  From  these  results,  its  evident 
that  those  questions  whose  standard  deviations  went  up  between  rounds  one  and  two  had 
reached  stability.  Further,  those  questions  whose  standard  deviation  went  down  had  not 
stabilized.  Therefore,  based  on  the  fact  that  stability  had  not  been  achieved  in  five  items 


78 


and  that  eight  out  of  14  questions  selected  for  the  third  questionnaire  were  new  to  the 
second  round  and  therefore  had  only  been  asked  once,  it  was  decided  to  proceed  with  the 

third  round. 

All  comments  provided  by  the  experts,  if  their  previous  response  to  a  question  was 
above  or  below  1.25  standard  deviations,  were  then  typed  into  a  separate  Ust.  The  specific 
comment,  expert,  and  whether  their  previous  answer  was  above  or  below  the  prescribed 
range,  was  annotated  on  the  list.  This  made  it  easier  to  group  similar  ideas  and  find 
common  themes. 

As  previously  mentioned,  the  lasft  objective  of  round  two  was  to  again  use  open- 
ended  questions  to  obtain  from  the  experts  any  additional  items  they  thought  might  be 
important  to  include  in  the  model.  However,  during  this  round  only  one  respondent 
provided  any  suggestions.  After  faculty  review,  these  were  deemed  to  already  be  part  of 
the  model  and  had  been  already  been  tested  for.  Consequently,  no  additional 
consideration  was  given  to  this  issue. 

Round  Three 

The  third  round  had  three  objectives.  The  first  was  to  gain  fiirther  consensus  or 
stability  on  items  previously  tested  in  rounds  one  and  two.  The  second  was  to  show 
consensus  or  stability  on  items  added  to  the  instruments  as  a  result  of  expert  feedback  in 
round  two  and  the  third  objective  was  to  determine  the  highest  rated  items.  The  scale 
used  on  this  survey  was  the  same  five  point  Likert  Scale  used  in  the  first  two  rounds. 

Results 

The  raw  data  collected  from  this  questionnaire  was  entered  and  reduced  in  the  same 
manner  as  the  second  survey.  However,  since  another  questionnaire  was  not  planned,  any 
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calculations  used  to  determine  the  items  for  the  next  questionnaire  were  not  required.  A 
listing  of  the  23  questions  asked  in  round  three  is  presented  in  a  rank  order  format  in  Table 

12. 
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Table  12.  Third  Round  Data  Sorted  Based  on  Mean 


Analysis  and  Findings 

As  previously  mentioned,  standard  deviation  and  IQR  are  two  common  measures 
of  consensus.  Consequently,  these  statistics  were  computed  for  each  of  the  items  in  this 
round  and  plotted  along  with  the  same  data  from  round  one  and  two  as  shown  in  Figure 
1 1  and  12.  As  seen  in  Figure  13  below,  the  general  trend  for  the  standard  deviations  from 
round  two  to  three  did  not  increase  as  they  did  from  round  one  to  two.  However,  they 
only  exhibited  a  slight  decrease. 


Figure  13.  Standard  Deviations  for  Rounds  1-3 


In  an  attempt  to  explain  this,  the  stability  of  each  item  was  again  considered  to 
determine  if  the  predictions  had  solidified.  Similar  to  Table  1 1,  Table  13  displays  the 
stability  trends  from  round  one  to  two  and  two  to  three.  Note  that  item  19  is  no  longer 
listed.  During  final  review  it  was  determined  that  the  wrong  question  was  attached  to  this 
items  feedback  statistics  in  the  second  and  third  round  questionnaires.  Therefore,  it  was 
discarded.  A  closer  analysis  of  this  issue,  however,  did  suggest  that  the  Delphi  process 
was  at  work.  This  issue  will  be  discussed  more  fully  in  the  next  chapter. 
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Table  13.  Stability  Measurements  for  Survey  Three  Questions 


Question 

Stability  between 
Rounds  1  and  2 

Std  Dev  (up  or 
down)  between 
rounds  1  and  2 

stability  between 
Rounds  2  and  3 

Std  Dev  (up  or 
down)  between 
rounds  2  and  3 

1 

14.29% 

Up 

14.29% 

Down 

3 

14.29% 

Up  . 

14.29% 

Up _ 

7 

14.29% 

Up 

14.29% 

Down 

11 

28.57% 

Up 

14.29% 

Up _ 

14 

14.29% 

Up 

0.00% 

Up _ 

23 

14.29% 

Up 

14.29%  ^ 

Down 

26 

0.00% 

Up 

14.29% 

Equal 

40 

42.86% 

Down 

14.29% 

Down 

43 

14.29% 

Up  ^ 

14.29% 

Down 

66  n 

28.57% 

Down 

14.29% 

Down 

73 

28.57% 

Down 

14.29% 

Down 

77 

28.57% 

Down 

14.29% 

Down 

80 

14.29% 

Down 

42.86% 

Down 

81 

14.29% 

Up _ 

14.29% 

Down 

As  shown  in  Table  13,  eight  out  of  nine  items  whose  standard  deviation  went 
down,  all  three  items  whose  standard  deviations  went  up,  and  one  item  whose  standard 
deviation  remained  the  same  aU  had  stabilities  lower  than  15%.  These  results  help  show 
fhat  the  responses  from  the  expert  group  had  stabilized  and  that  consensus  had  been 
achieved.  This  argument  is  reinforced  by  the  fact  that  all  interquartile  range  values  were 
one  or  lower  in  roimd  three,  as  seen  if  Figure  14. 
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Figure  14.  Interquartile  Range  for  Rounds  1-3 


This  decrease  also  signifies  that  consensus  has  been  reached  for  these  items  (Schiebe, 
1975). 

As  previously  mentioned,  the  second  objective  of  round  three  was  to  show 
consensus  or  stability  for  items  added  to  the  instmments  as  a  result  of  expert  feedback  in 
round  two.  Figure  15  shows  the  standard  deviation  of  these  items  fi-om  roimd  two  to 
three. 
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Figure  15.  Standard  Deviations  for  New  Items  between  Rounds  2-3 


Clearly,  the  trend  for  these  items  is  downward,  again,  showing  the  drive  towards  a 
perfect  consensus  number  of  zero  (Dickson,  1984).  This  assessment  is  reinforced  by  the 
changes  in  the  interquartile  ranges  for  these  items  as  shown  in  Figure  16  below. 


Figure  16.  Interquartile  Range  for  New  Items  between  Rounds  2-3 
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Again,  this  plot  also  shows  a  downward  trend.  In  fact,  aU  these  items  have  an  IQR  of  less 
than  one,  which  could  indicate  consensus  had  already  been  formed.  However,  the  stability 
numbers  for  these  items  listed  in  Table  14  do  not  support  that  viewpoint. 

Table  14.  Stability  of  New  Questions 


New  Question 

Stability 

N85 

42.86% 

N88 

0.00% 

N89 

14.29% 

N92 

42.86% 

N93 

42.86% 

N95 

57.14% 

N96 

14.29% 

N97 

28.57% 

With  five  out  of  eight  stability  measurements  about  15%,  it  is  difficult  to  state  that 


stability  has  been  achieved  for  these  items. 

Round  three’s  final  objective  was  to  determine  the  highest  rated  items  in  the  study. 
The  top  ten  highest  rated  items  are  shown  in  Table  15.  These  items  were  generated  by 
applying  a  threshold  value  of  4. 142  to  the  results  fi-om  round  three.  Only  ten  were 
selected  since  it  is  a  small,  convenient  sample  which  most  people  are  accustomed  to 


dealing  with. 
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Table  15.  Top  Ten  Highest  Rated  Items 


Question 

Number 

Rank 

Question 

Average 

Item 

Type 

23 

1 

Improve  quantity  or  quality  of  agency  outputs 

B1 

26 

1 

Productivity  improvements 

gIdrM 

B1 

N96 

1 

I  believe  outsourcing  should  not  be  done 
without  a  comprehensive  business  assessment 
to  establish  value 

F5 

73 

4 

I  believe  some  IT  managers  outsource  to 
eliminate  a  troublesome  function  that  is  "not 
worth  the  headaches" 

4.429 

P4 

N95 

5 

I  believe  the  cost  of  outsourcing  is  not  clearly 
understood 

4.286 

F5 

77 

5 

I  believe  some  IT  managers  attempt  to 
convince  senior  managers  that  their  internal 

IT  departments  can  perform  the  same 
services  an  outside  vendor  can  perform  for 
the  same  amount  or  less 

4.286 

PI 

3 

5 

Improve  technical  services  (e.g. 
dissatisfaction  with  current,  in-house  IT 
support) 

4.286 

T1 

43 

8 

Security  concerns  over  how  vendors  handle 
agency  data  on  their  own  systems 

4.143 

S2 

81 

8 

I  believe  the  more  other  government  agencies 
outsource  their  IT,  the  more  mine  might 

4.143 

E2 

N92 

8 

I  believe  outsourcing  activities  are  not 
initiated  by  IT  organizations.  External  actors 
loom  large  in  decisions 

4.143 

NE2 

Summary 

This  chapter  has  detailed  what  the  highest  ranked  items  in  the  question  were,  how 
they  where  determined  and  whether  consensus  on  them  was  achieved.  The  next  chapter 
will  use  these  results  to  investigate  any  relationships  between,  and  within  the  factors. 
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V.  Findings  and  Conclusions 

Introduction 

This  chapter  is  broken  into  three  sections.  The  first  provides  a  macro  perspective 
of  the  results  by  discussing  how  each  factor  relates  to  the  others.  The  second  portion 
provides  a  micro  perspective  by  focusing  within  individual  factors  and  attempts  to  ejq)lain 
and  reconcile  the  results  based  on  the  comments  received  fi-om  the  expert  group  in  the 
second  survey.  The  third  section  lists  any  limitations  of  the  research  and  suggests  possible 
follow  on  activities. 

Macro  Perspective 

When  considering  the  importance  of  the  fectors  relative  to  each  other,  it  is  helpful 
to  consider  the  percentage  of  questions  retained  fi'om  the  first  round  to  the  last.  This 
measure  gives  a  crude  estimate  of  what  factors  are  most  important.  Table  16  summarizes 
this  information  by  presenting  how  many  questions  were  associated  with  each  factor  and 
how  many  were  retained  ifor  the  last  round.  The  percentages  fiDr  each  were  then 
calculated  and  ranked.  The  total  number  of  questions  listed  was  determined  by  adding  the 
total  number  of  original  questions  plus  any  additional  items  suggested  by  the  experts.  It 
should  be  noted  that  the  entirely  new  factor  best  practices  and  improved  process  (NAl) 
was  not  considered  in  this  table.  For  clarification  purposes.  Table  17  lists  the  number  of 
first  round  questions  along  with  the  specific  questions  added  and  their  assigned 
alphanumeric  designator. 
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Table  16.  Number  of  Questions  Retained  from  Round  One  to  Three 


FACTORS 

First  to  Third  Round 

Number  of  Questions 

Retained 

(Percentage  retained) 

Ranking  of  %  retained 

Financial 

17 

7  (41%) 

1 

Business 

35 

4  (13%) 

6 

Technical 

14 

3  (21%) 

4 

Political 

12  1 

3(25^ 

3 

Security 

10 

2  (20%) 

5 

Environmental 

8 

3  (38%) 

2 

Table  17.  Number  of  Questions 


Factor 

Number 

of 

questions 
in  round  1 

Additional  questions  included  in  round  2 

Final  total 
number  of 
questions 
asked 

Financial 

13 

1.  Create  agency  wide  business  practice  (F2) 

2. 1  believe  the  cost  of  outsourcing  is  not  clearly 
understood  (F5) 

3. 1  believe  outsourcing  should  not  be  done  without  a 
comprehensive  business  assessment  to  establish  value  (F5) 

4. 1  believe  managers  do  not  determine  best  value  prior  to 

outsourcing  (F5) _ _ 

17 

Business 

30 

1.  Congressional  Interest  (B2) 

2.  Lack  of  a  clearly  delineated  set  of  systems,  costs  and 
service  levels  (B4) 

3. 1  believe  that  many  agencies  feel  that  outsourcing  is 
either  mandated  or  so  strongly  encouraged  as  to  be,  in 
essence  mandated  (B2) 

4. 1  believe  that  many  senior  IT  managers  see  outsourcing 
as  an  end  in  itself  rather  than  a  strategy  to  achieve  a  goal 
(B2) 

5. 1  believe  some  IT  managers  view  outsourcmg  as  a 
means  to  mitigate  the  inability  of  government  to  remove 
incompetent  IT  workers  (B4) 

35 

Technical 

13 

1.  Ability  of  IT  outsourcing  to  make  interoperability 
between  systems  LANs  better  (T3) 

14 

Political 

12 

No  questions  added 

12 

Security 

9 

1.  Controlling  different  security  levels  of  data  without 
security  incident  (S2) 

10 

Environmental 

6 

1.  (Quality  of  Intra-agency  relationships  (between  IT  and 
business  units)  (NEl) 

2. 1  believe  outsourcmg  activities  are  not  initiated  by  IT 
organizations.  External  actors  lomn  large  in  decisions 
(NE2) 

8 

***The  question  “I  believe  some  IT  managers  view  outsourcing  as  a  means  to  adopt  industry 
_ best  practices  and  improve  processes  (NAl)”  was  not  included  *** - 
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From  Table  16,  it  seems  that  financial  factors  were  the  most  important 
determinants  to  the  population.  This  result  is  not  surprising  considering  some  have  labeled 
this  factor  the  most  important  advantage  provided  by  outsourcing  (Rebeiro,  1996). 
Additionally,  its  been  shown  that  reduced  costs  are  one  of  the  largest  motivators  behind 
the  use  of  IT  outsourcing  in  the  government  (Jones,  1 999).  As  mentioned  in  chapter 
two,  cost  reductions  were  repeatedly  labeled  the  number  one  reason  why  outsourcing 
solutions  should  be  pursued  during  the  interviews  conducted  for  this  research.  In  fact,  one 
respondent  casually  remarked,  “is  there  another  reason  you  would  outsource?”  (Folsom, 
2000).  A  government  Chief  Information  Officer  (CIO)  has  even  been  quoted  that  when  it 
comes  to  outsourcing,  “cost  is  THE  driver”  (Jones,  1999:29). 

The  second  most  important  factor  seemed  to  be  environmental  determinants,  while 
the  third  were  the  political  items.  Considering  the  political  nature  of  any  public  firm,  this 
result  is  not  surprising.  Additionally,  the  feet  that  these  two  factors  were  ranked 
successively  might  suggest  that  they  are  closely  related.  It  is  possible  that  since  the 
environment  that  public  firms  operate  in  is  inherently  political,  there  might  be  a  correlation 
between  these  two  fectors.  In  a  close  tie  for  four  and  fifiih  place  are  technical  and  security 
items  respectively.  However,  the  lower  rankings  of  these  items  should  probably  not  be 
viewed  as  a  lack  of  concern  for  such  issues.  As  an  example,  expert  #6  ejqiressed  his 
organizations  technical  focus  by  commenting,  “our  concern  is  with  providing  availability 
of  IT  services  -  we  seek  99.975%  network  availability.  We  also  want  to  increase  overall 
desktop  availability  to  99.975%  for  average  users  and  99.99%  for  executive  users.”  Such 
goals  seem  to  be  consistent  with  the  wishes  and  desires  of  many  other  DoD  organizations 
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(AFIT,  2000).  These  users  seem  to  feel  that  the  first  priority  of  any  DoD  IT  organization 
should  be  to  provide  network  service  that  is  as  reliable  as  a  telephone. 

The  least  important  fector  seemed  to  be  business.  If  this  result  seems  surprising, 
one  could  argue  that  it  was  simply  because  the  business  factor  had  more  sub-factors  to  test 
for  and  consequently  had  more  that  could  be  rejected.  A  good  example  of  this  are  the  10 
re-tested  items  that  were  aU  rejected  in  the  second  round.  However,  Table  18  shows  that 
even  when  these  1 0  items  are  not  considered,  the  business  factor  did  not  rate  much  higher. 
Also,  as  will  be  shown  later,  while  the  business  factor  as  a  whole  did  not  rate  very  high, 
several  individual  business  sub-factors  did. 

Table  18  presents  a  more  detailed  look  of  the  same  data  by  delineating  how  many 
questions  were  originally  associated  with  each  factor  in  the  first  and  second  round  as  well 
as  how  many  were  retained  for  the  next.  The  percentages  for  each  were  then  calculated 
and  ranked.  As  in  Table  16,  the  number  of  questions  listed  in  the  second  to  third  round 
includes  any  additional  items  or  sub-factors  added  by  the  experts.  Also,  as  noted  in  the 
chapter  four,  the  10  Business  related  questions  that  needed  to  be  rephrased  and  retested 
fi'om  the  first  to  second  roimds  are  not  included  since  there  is  difficulty  determining  which 
round  to  assign  them  to.  This  table  also  does  not  include  the  single  additional  fector 
tested  for  in  the  second  round  relating  to  best  practices  and  policies. 
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Table  18.  Number  of  Questions  Retained  from  Round  One  to  Two  and  Two 

to  Three 


FACTORS 

First  to  Second  Round 

Second  to  Third  Rotmd 

Number  of 
Questions 

Retained 

(Percentage 

retained) 

Ranking  of 
%  retained 

Number  of 
Questions 

Retained 

(Percentage 

retained) 

Ranking  of 
%  retained 

Financial 

13 

9(  69%) 

3(tie) 

13 

7  (54%) 

1 

Business 

20 

10  (50%) 

5 

15 

5 

Technical 

13 

9  (69%) 

Bin 

10 

4 

Political 

12 

6 

3 

2 

Security 

9 

2 

8 

6 

Environmental 

6 

1 

7 

3 

*  10  Business  factors  retested 

*A11  retested  business  fectors  Avere  rejected 

As  shown,  while  the  rankings  vary  slightly  from  Table  16,  they  are  consistent  with  the 
results  presented  there.  Business  determinants  still  rank  among  the  lowest  items. 
However,  it  should  be  noted  that  the  last  three  determinants  all  fall  within  5%  of  each 
other,  helping  to  show  that  they  are  equally  important. 

Another  reason  why  business  determinants  may  have  ranked  so  low  is  because  firms 
seem  worried  they  might  lose  critical  skill  sets  and  therefore  a  competitive  edge  over  their 
adversaries.  As  stated  by  expert  #2,  “It  is  absurd  to  believe  future/long  term  issues  will 
not  require  understanding  and  recognition  that  IT  is  an  essential  tool  for  the  company  s 
future  well  being.”  He  also  seems  to  believe  that  IT  will  not  simplify  the  management 
agenda  since  any  difficult  decision  regarding  IT  will  still  need  to  be  made  internal  to  the 
firm  As  he  states: 

Handing  it  [IT]  over  to  an  outsourcing  firm  does  not  absolve  the  company 
from  making  tough  decisions.  I  don’t  think  the  head  of  an  outsourced  firm 
will  be  able  to  simplify  key  decision  making  issues  any  easier  than  a  good 
internal  IT  manager  who  is  respected  by,  and  embraced  as  part  of  the 
senior  management  team. 


Expert  #10  echoes  this  feeling  when  he  states,  “The  level  of  management  that  manages  the 
IT  organization  (whether  in-house  or  outsourced)  would  likely  remain  the  same.  Why 
would  someone  else  doing  the  work  change  the  management  agenda?’ 

It  is  also  interesting  to  note,  that  all  factors  are  strongly  represented  in  Table  16. 
This  might  suggest  that  while  some  determinants  are  more  important  than  others,  the 
decision  to  outsource  IT  in  the  DoD  is  a  multifaceted  one.  This  viewpoint  is  even 
supported  when  only  the  ten  highest  rated  determinants  are  considered.  As  seen  in  Table 
15,  at  least  one  item  from  each  factor  was  represented  in  this  list,  again  demonstrating  the 
convoluted  nature  of  this  decision.  The  multidunensional  character  of  these  results  is  also 
consistent  with  previous  studies.  As  mentioned  in  chapter  two,  most  of  the  existing  IT 
outsourcing  frameworks  for  the  private  sector  also  have  a  multidimensional  flavor  (Loh, 
1992;  Smith,  1998;  Gurbaxani,  1996;  Lacity,  1994).  Perhaps  the  results  of  this  study  are 
an  indication  that  private  and  public  sector  organizations  tend  to  have  similar  IT 
outsourcing  determinants. 

Micro  Perspective 

Each  fector  weis  also  examined  internally  for  any  trends.  By  focusing  on  each 
factor  individually,  patterns  begin  to  emerge  that  provide  clues  as  to  why  certain  experts 
may  have  voted  they  way  they  did.  In  addition,  it  allows  a  more  comprehensive  analysis 
of  the  issues  and  concerns  surroxmding  each  factor.  Expert  comments  were  used  to  help 
illustrate  these  issues  and  results. 

Financial 

Table  19  shows  the  average  for  each  of  the  questions  in  the  last  round  along  with 
their  ranking  in  each.  As  shown,  two  of  the  final  top  rated  items  were  financial 
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determinants.  These  were  I  believe  outsourcing  should  not  be  done  without  a 
comprehensive  business  assessment  to  establish  value  (N96)  and  I  believe  the  cost  of 
outsourcing  is  not  clearly  understood  (N95).  Both  of  these  items  had  consistently  high 
averages  throughout  the  entire  study.  It  is  interesting  to  note  that  both  came  from  the 
same  sub-factor,  potential  problems  with  cost  reductions  (F5). 
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Table  19.  Ranking  of  Final  Items  in  Each  Round 
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Based  on  this,  it  seems  that  DoD  IT  officials  are  cautious  about  the  potential  for 

any  cost  savings  from  outsourcing.  This  seems  to  be  inconsistent  with  previous  studies, 

which  seemed  to  suggest  that  IT  outsourcing  was  performed  in  the  Government  almost 

exclusively  for  cost  related  reasons  (Jones,  1999).  This  apparent  contradiction  might  be 

explained  by  the  timing  of  each  of  the  studies.  As  a  comparison,  many  private  firms  also 

tended  to  focus  on  cost-related  determinants  early  on  in  their  experiences  with  IT 

outsourcing  (Rebeiro,  1996).  However,  as  their  femiliarity  with  it  grew  they  seemed  to 

become  more  skeptical  over  any  potential  savings.  (Lacity,  1993;  Quinn,  1999;  Willcocks, 

1994).  The  DoD  might  simply  be  experiencing  the  same  evolution.  Recent  IT 

outsourcing  studies  in  the  Federal  Government  show  that  cost  savings  occurred  for  only 

the  first  year  or  two  and  that  no  evidence  exists  for  savings  due  to  economies  of  scale 

(Jones,  1999).  This  idea  is  corroborated  by  the  fact  that  many  DoD  agencies  now  seem  to 

be  performing  Total  Cost  of  Ownership  (TCO)  studies,  which  will  hopefully  help  prove 

any  potential  cost  saving  before  a  decision  is  made  to  outsource  (Cooper,  2000;  NSA, 

1999).  This  approach  is  also  being  endorsed  by  the  senior  managers  in  several  private 

firms  who  believe  that  to  truly  determine  whether  outsourcing  vwU  produce  cost  savings, 

all  information  management  activities  costs  must  be  known  (Martinsons,  1993). 

Identifying  IT  costs  is  not  the  only  financial  objective  sought  by  organizations. 

They  also  seem  to  be  starving  for  any  process  that  will  provide  some  predictability  to  their 

IT  budgets.  Some  feel  that  outsourcing  can  provide  this.  As  expert  #6  writes: 

In  our  organization,  the  true  cost  of  operating  our  IT  is  often  masked  by 
various  business  units  using  mission  money  for  IT  activities.  Applications 
are  stood  up  by  functional  users  and  turned  over  for  maintenance  to  the  IT 
shop.  The  IT  shop  had  no  knowledge  of  this  activity  and  had  not  budgeted 
for  maintenance  of  the  servers,  software  licenses,  etc.  In  a  similar  vein. 
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business  units  have  bought  PDA  software,  and  other  equipment,  not  on 
configuration  but  with  the  ejqjectation  of  help  supporting  these  items. 
Outsourcing  would  require  all  IT  fimctions  to  be  procured  and  supported 
through  the  contractor.  These  costs  would  no  longer  be  a  surprise  to  the  IT 
staff.  Business  units  would  have  to  bear  the  financial  burden  of  their  IT 
decisions. 

However,  there  are  those  who  seem  to  feel  that  outsourcing  is  only  treating  the  symptoms 

and  not  curing  this  disease.  As  expert  #10  suggests,  “it  is  really  the  internal  organization 

that  has  to  change.”  To  support  this  position  expert  #2  states: 

Improving  cost  controls  MAY  be  an  outcome  of  outsourcing,  but  it  is  not 
the  most  important  consideration  for  making  an  outsourcing  decision.  If  an 
organization  does  not  understand  how  outsourcing  will  REQUIRE  change 
to  established  business  practices  first,  costs  may  not  be  controlled  any 
better  than  before  the  outsourcing.  In  fact,  this  has  been  the  case  for  some 
corporations  who  did  not  first  consider  the  more  essential  question:  will 
you  allow  an  outsourced  firm  to  discipline  your  workforce’s  IT 
consumption? 

Expert  #9  also  seems  to  express  this  opinion  by  stating,  “This  objective  should  be 
accommodated  via  fiscal  means  and  not  through  outsourcing.” 


Business 

As  mentioned  earlier,  the  business  fector  as  a  whole  did  not  rank  very  high. 
However,  as  seen  in  Table  15,  two  of  the  three  highest  rated  items  were  business 
determinants.  In  fact,  as  shown  in  Table  19,  these  two  items  remained  at  or  near  the  top 
of  the  ranking  in  all  three  roimds.  It  is  also  interesting  to  note  that  both  were  fi-om  the 
focus  on  core  competency  (FI)  business  sub-factor,  suggesting  that  while  business  factors 
are  not  considered  very  much  as  a  whole,  this  sub-factor  is  considered  heavily  when 
making  a  decision  to  outsource. 

It  is  interesting  to  note  that  previous  research  has  shown  that  DoD  organizations 
do  not  consider  business  determinants  much  at  aU  when  making  an  IT  outsourcing 
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decision,  but  focus  exclusively  on  financial  determinants  (Jones,  1999).  After  all,  the 
financial  factor  did  rank  highest.  Yet  these  results  would  tend  to  suggest  otherwise.  As  in 
the  similar  financial  situation  above,  this  apparent  discrepancy  might  be  explained  by  the 
fact  that  many  private  firms  also  tended  to  focus  on  financial  determinants  early  on  in  their 
experiences  with  outsourcing  (Rebeiro,  1996).  But,  as  their  familiarity  with  it  grew,  they 
seemed  to  start  thinking  more  about  the  business  related  aspects  of  the  arrangement 
(Venkatraman,  1997).  Again,  the  DoD  might  simply  be  experiencing  this  same  evolution. 
This  idea  is  further  strengthened  when  it  is  considered  that  focusing  on  core  competencies 
was  among  the  first  business  determinants  considered  by  private  industry. 

Based  on  the  comments,  there  also  seems  to  be  a  deep  concern  with  a  potential 

loss  of  control  over  IT  services.  As  expert  #2  states: 

This  is  a  big  issue  for  US  Intelligence.  There  is  a  real  possibility  that  the 
government  may  lose  the  expertise  to  understand  technology  and  related 
insight  into  security  issues.  The  government  may  lose  key  expertise  of  how 
other  nations  may  use  technology,  and  thereby,  what  vulnerabilities  exist 
for  US  and  foreign  systems.  This  factor  demands  up  front  consideration 
over  what  to  outsource  and  what  not  to  outsource;  what  the  legal  aspects 
may  be  as  related  to  national  security  if  a  company  owns  aspects  of  the  IT 
infrastructure  (what  are  the  government’s  rights  to  control  data);  what 
happens  when  a  foreign  company  buys  out  an  outsourcing  firm  that  has 
access  to  national  security  information;  how  the  government  will  retain  key 
skills,  etc. 

Yet,  there  are  some  who  feel  that  this  is  only  a  temporary  concern,  as  expert  #10  states, 
“It’s  a  competitive  market  now,  but  it  ebbs  and  flows.  This  may  be  the  case  now  but  not  a 
year  or  two  from  now.” 


Technical 

Improving  technical  services  also  rated  high  in  the  survey.  As  seen  in  Table  19, 
question  26,  Productivity  improvements,  ranked  fairly  high  throughout  in  aU  three  surveys. 
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This  item  concentrates  on  improving  technical  services.  However,  some  feel  that 
improving  services  is  not  the  issue.  Rather,  the  issue  is  improving  them  within  the  budget 
allotted.  As  expert  #10  wrote,  “Our  main  headache  is  finding  quahty  IT  workers  that  we 
can  afford.”  Consequently,  it  seems  that  qualified  workers  that  can  increase  customer 
satisfaction  are  available,  although,  paying  for  them  is  another  issue. 

Yet,  there  are  some  who  feel  that  any  increased  service  levels  provided  by 
outsourcing  are  overstated.  As  expert  #10  wrote,  “Frequently  these  are  beliefs  not  based 
on  actual  data.  [It  is]  tough  to  change  beliefs.”  Based  on  this,  it  seems  that  some 
managers  and  users  may  simply  trust  what  they  have  heard  about  IT  outsourcing  rather 
than  investigating  it  themselves.  Expert  #2  echoed  this  position  by  statmg,  “Customers 
may  not  be  any  happier,  possibly  less  so,  when  an  outsourcing  firm  has  total  control  over 
their  IT  environment  and  begins  to  discipline  IT  customers.  They  may  have  less  recourse, 
other  than  to  grumble.” 

Additionally,  some  seemed  concerned  that  vendors  don’t  have  the  ability  to  solve 
many  agency  specific  requirements.  As  expert  #7  writes,  “One  of  the  key  issues  that 
would  inhibit  one  fi-om  outsourcing  would  be  concern  that  a  vendor  could  not  meet  the 
unique  requirements  (often  security  related)  of  my  organization.”  As  he  points  out,  many 
of  these  special  requirements  are  security  related.  This  is  not  surprismg  given  the 
classified  nature  of  the  data  these  organizations  sometimes  handle.  Others  seem  to 
counter  this  claim  by  proposing  that  a  properly  written  contract  can  help  minimize  these 
issues.  As  expert  #  9  writes,  “many  of  the  [technical]  complications  can  be  mitigated  via 
contract  provisions.”  In  the  end,  it  seems  that  some  are  unconvinced  that  outsourcing  IT 
can  increase  user  satisfaction. 
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It  also  seems  that  the  influence  these  issues  have  on  a  decision  to  outsource 
depends  in  large  measure  as  to  whether  the  organization  making  it  is  a  policy  or 
operational  group.  As  expert  #6  states,  “We  are  a  policy  shop.  We  do  not  have  the  large 
number  of  integration  and  system  dependencies  that  an  operational  environment  such  as 
Army  Personnel  Command  would  have.” 

Political 

There  were  also  several  political  questions  that  seemed  important  to  the  expert 
group.  As  seen  in  Table  19,  questions  77, 1  believe  some  IT  managers  attempt  to 
convince  senior  managers  that  their  internal  IT  departments  can  perform  the  same 
services  an  outside  vendor  can  perform  for  the  same  amount  or  less,  and  73, 1  believe 
some  IT  managers  outsource  to  eliminate  a  troublesome  function  that  is  “not  worth  the 
headaches",  both  ranked  high  in  all  three  rounds.  The  first  question  centers  on  using  IT 
outsourcing  proposals  to  prove  to  senior  managers  that  when  compared  directly,  the 
internal  team  can  perform  just  as  good  if  not  better  than  an  outside  vendor.  As  expert  #10 
wrote: 

The  fact  of  outsourcing  being  better  AND  cheaper  may  hold  true  for  a 
short  period  of  time.  Constraints  on  the  existing  organization  will  also 
constrain  an  outsourced  organization.  So,  if  you  reduce  the  constraints, 
internal  organizations  can  perform  better. 

As  mentioned,  many  times  the  internal  team  is  handicapped  by  not  being  allowed  to 
compete  using  the  same  requirement  set.  As  the  literature  has  pointed  out,  “many  IT 
departments  have  equally  sophisticated  technology  and  adequate  economies  of  scale  but 
aren’t  allowed  to  adopt  the  best  practices  that  would  help  them  match  or  beat  a  supplier  s 
bid”  (Lacity,  1995:  90).  It  goes  on  to  state  that  if  a  company  allows  its  own  IT 
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department  to  compete  with  outsourcing  vendors,  they  not  only  might  win  the  contract 
but  can  also  “gain  a  much  deeper  understanding  of  the  costs  of  a  given  service  and  the 
best  way  to  provide  it.  If  they  decide  to  outsource  in  the  future,  they  will  be  in  a  stronger 
position  to  evaluate  bids  and  to  write  a  contract  that  serves  their  own  interests”  (Lacity, 

1995:  91) 

Question  73, 1  believe  some  IT  managers  outsource  to  eliminate  a  troublesome 
function  that  is  “not  worth  the  headaches",  also  rated  high.  This  items  focuses  on  an  IT 
managers  desire  to  simply  rid  themselves  of  a  difficult  and  problematic  activity.  This  issue 
is  anchored  to  the  idea  that  IT  can  be  a  very  demanding  and  sometimes  unrewarding 
activity.  Many  times  IT  administrators  do  not  adequately  feel  conqiensated  either 
emotionally  or  financially  when  they  have  success.  In  addition,  they  feel  overly  targeted 
by  users  when  something  goes  wrong  (Lacity,  1994).  This  can  lead  to  IT  managers  simply 
outsourcing  the  problem  to  someone  else.  Some  feel  that  this  approach  can  also  have  the 
added  benefit  of  focusing  the  attention  of  an  organizations  senior  leadership  on  IT.  As 
expert  #2  wrote. 

Some  managers,  both  in  IT  and  at  the  corporate  level  would  rather  toss  the 
problem  over  the  fence.  Have  you  heard  the  term,  “IT  is  not  part  of  our 
core  mission?’  This  also  happens  when  an  IT  manager  cannot  get 
corporate  support  for  making  tough  decisions.  By  outsourcing  the  issue, 
they  force  the  organization  to  recognize  “troublesome”  functions  can  be 
tough  to  resolve  whether  run  by  the  government  or  contractors. 

The  viewpoint  expressed  by  eiqiert  #10  on  this  issue  seems  even  more  basic  when  he 

writes,  “I  see  outsourcing  as  a  management  strategy-which  would  include  ebminating  a 

troublesome  function.”  From  this  it  seems  he  believes  that  eliminating  a  troublesome 

function  is  simply  a  beneficial  by-product  of  the  proper  overall  management  strategy. 
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During  the  initial  interviews  it  also  became  clear  that  exposing  exaggerated  clauns 
was  a  potential  reason  to  outsource.  In  one  interview,  the  CIO  of  a  DoD  agency 
mentioned  that  there  is  big  push  towards  outsourcing  IT  within  the  government  (Cooper, 
2000).  This  is  not  surprising  considering  Congressional  language  exists  stating  that 
Government  agencies  shall  outsource.  However,  some  agency  CIOs  have  not  been 
convinced  that  outsourcing  is  the  panacea  others  might  think  it  is.  Some  of  these 
individuals  have  been  successful  at  convincing  Congressional  staffers  to  allow  their 
organization  to  perform  outsourcing  evaluations  to  determine  exactly  how  much  might 
truly  be  saved  if  outsourcing  were  pursued. 

Security 

As  theorized,  security  was  also  represented  in  the  top  ten  list  of  determinants. 
Several  experts  emphasized  the  fact  that  security  was  an  important  factor  in  making  a 
decision  to  outsourcing  their  IT.  Expert  #6  wrote,  “We  operate  in  a  highly  sensitive  and 
political  environment.  Who  has  access  to  our  data  is  a  concern.  Expert  #8  echoed  this 
sentiment  by  stating,  “[We]  operate  in  a  high  security  environment.  Every 
vendor/contractor  must  be  cleared  to  this  level.”  Expert  #6’s  comment  is  also  interesting 
because  it  points  out  that  the  environment  is  not  just  sensitive,  but  political  as  well.  This 
reveals  that  enemies  of  the  state  may  not  be  the  only  adversaries  individuals  attempt  to 
conceal  information  from.  As  mentioned  in  chapter  two,  individuals  in  public  sector 
organizations  can  sometimes  base  decisions  on  more  personally  motivated  criteria 
(Hancox,  1999). 

Yet  when  it  comes  to  protecting  sensitive  information,  other  experts  feel  that 
contract  provisions  and  existing  poUcy  are  adequate  safeguards.  As  expert  #2  writes. 
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Not  a  key  decision  fector.  Rules  are  written  into  any  contract  that  will 
specify  how  government  information  is  to  be  treated.  This  may  require  a 
company  to  change  their  practices.  Put  the  onus  on  the  vendor,  make  the 
vendor  support  the  government’s  requirement  for  privacy  and  security  as 
part  of  a  selection  process.  Do  not  make  this  an  up-front  decision— the 
extent  they  currently  support  adequate  privacy. 

Expert  #2  follows  this  by  stating,  “the  security  screening  mitigates  this  concern;  once  they 

have  passed”  and  expert  #9  wrote,  “We  maintain  the  highest  level  of  mtegnty.  Any 

security  concern  would  be  given  the  utmost  attention. 

This  reliance  on  contract  clauses  to  minimize  security  problems  is  endorsed  by  the 

literature.  As  mentioned  in  chapter  two,  there  are  those  who  have  proposed  that  the 

security  framework  used  when  IT  work  is  outsourced,  is  different  than  when  the  work  is 

kept  in-house  (Fink,  1994)  and  that  security  issues  for  IT  outsourcing  agreements  are 

different  than  for  most  other  normal  commercial  contracts  (Rebeiro,  1996).  Therefore,  it 

is  important  that  those  negotiating  the  contracts  understand  this  so  they  can  structure  the 

contract  to  best  support  an  organizations  security  requirements.  To  help  assist  them,  a 

systems  approach  is  sometimes  advocated,  whereby,  the  entire  security  framework  is  first 

defined  and  analyzed.  FoUowing  this,  appropriate  contract  provisions  can  be  written  to 

support  the  overaU  strategy  (Rebeiro,  1996).  These  provisions  might  include  specific 

performance  measures  that  can  be  assessed  and  tested  including  response  times,  systems 

availability,  data  security  and  privacy/user  authorization  (Fink,  1 994). 

Environmental 

When  considering  the  influence  of  the  environment  on  IT  outsourcing  decisions, 
drSlision  theory  and  external  actors  seem  to  take  center  stage.  As  mentioned  in  chapter 
two,  IT  outsourcing  can  be  considered  an  administrative  innovation.  As  such,  it  is 
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difiused  or  communicated  through  certain  channels  over  time  among  the  members  of  the 
firm’s  social  system.  Of  the  two  types  of  difiusion  models,  internal  and  external,  the 
internal  version  seems  most  applicable  in  this  situation  since  it  applies  when  organizations 
are  their  sourcing  decision  based  on  other  organizations  that  have  already 

outsourced  (Loh,  1992).  This  model  is  based  on  the  coefficient  of  internal  influence, 
which  is  the  likelihood  that  somebody  who  is  not  yet  using  the  process  will  start  using  it 
because  of  word-of-mouth  or  other  influence  fi'om  those  already  using  the  process 
(Mahajan,  1989;  Neflson,  1995).  Based  on  this,  the  closer  the  channels  of  communication 
are  linked,  and  the  more  homogenous  the  group  is,  the  quicker  and  more  powerful  the 
difiusion  will  be.  Since  military  related  organizations  are  usually  considered  very 
homogenous  and  well  connected,  it  is  not  surprising  internal  difiusion  theory  can  thrive 
within  it.  Therefore,  it  is  not  surprising  that  difiusion  was  rated  as  a  top  determinant  to  IT 
outsourcing.  This  entire  idea  is  captured  by  expert  #10  who  commented,  ’’there  is  a  ‘what 
is  eveiyone  doing-let’s  follow’  mentaUty  in  government.”  However,  there  are  others  who 
felt  the  opposite.  They  believed  that  difiusion  theory  has  only  a  small  affect  in  their 
organizations  because  IT  managers  showed  resistance  to  outsourcing.  As  expert  #  6 
wrote,  “resistance  to  our  outsourcing  has  come  fi'om  IT  managers  trying  to  protect  their 
fiefdom.”  Expert  #10  reinforces  this  idea  by  stating,  “human  nature  is  to  resist  change. 
Change  usually  occurs  when  motivated  by  an  outside  source.” 

The  second  top-rated  environmental  item  was  a  new  one  submitted  by  an  ejqiert  in 
round  one.  This  question  revolved  around  the  affect  of  external  actors  on  a  decision  to 
outsource.  Since  in  this  situation,  information  on  the  innovation  is  not  obtained  through 
interorganizational  communication  but  comes  fi'om  outside  the  system,  some  might  argue 
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that  it  could  be  represented  by  the  external  influence  model  (Loh,  1992).  However,  it 
seems  more  likely  that  it  is  singly  an  effect  of  the  hierarchical  structure  inside  the 
Government.  Since  no  comments  were  solicited  in  the  third  round  for  this  item,  it  is 
difficult  to  determine  exactly  what  was  meant  by  external  actors,  but  seems  reasonable, 
based  on  in  the  interviews  conducted  at  the  beginning  of  this  research,  that  the  actors 
being  referenced  are  senior  to  the  stakeholders.  Examples  might  include  Congressional 
members  or  senior  leadership  within  the  executive  branch. 

Additional  Factors 

One  additional  fector  created  by  the  expert  group  also  made  it  into  the  top  ten 
highest  ranked  items.  The  abihty  of  outsourcing  to  provide  a  method  to  adopt  industry 
best  practices  and  improved  processes  (NAl)  was  fisted.  Again,  since  no  comments  were 
solicited  in  the  third  round,  it  is  difficult  to  determine  exactly  what  was  meant  by  it. 
However,  one  possible  interpretation  is  that  IT  outsourcing  aUows  public  firms  the  ability 
to  leverage  the  experiences  and  success  of  private  firms  while  learning  fi:om  their  mistakes. 
Based  on  this  it  could  be  considered  as  a  business  related  sub-fector.  However,  when  the 
difiusion  of  these  process  among  the  members  of  a  firm  is  considered,  it  could  be 
considered  an  environmental  determinant. 

Limitation  and  Follow  on  Research 

Perhaps  the  biggest  limitation  of  this  research  was  expert  mortality.  It  has  been 
said  that  one  of  the  largest  problems  associated  with  the  Delphi  approach  is  mortality  and 
this  seems  no  truer  than  when  studying  IT  related  topics  using  semor  leaders.  The 
mortality  in  this  study  was  30%,  and  while  high,  could  have  been  even  higher.  During  this 
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research  two  experts  retired  and  two  others  took  new  jobs  yet  one  of  the  retirees  still 
participated  in  the  final  round.  This  turnover,  while  high,  is  not  surprising  considering  that 
the  average  duration  each  expert  in  this  study  had  held  his  position  was  less  than  one  year. 
Two  potential  reasons  for  this  high  turnover  and  inability  to  retain  qualified  senior  IT 
executives  in  the  government  might  be  the  current  climate  in  the  IT  commumty  and  the 
strength  of  the  economy.  It  is  difficult  for  the  Government  to  compete  with  lucrative 
offers  from  the  labor  hungry  private  sector  (Matthews,  2000).  In  an  attempt  to  combat 
these  forces  and  lure  IT  workers  into  government  service,  special  program  and  pay  scales 
have  even  been  created  for  them  (Brewin,  2000;  Robb,  2000). 

Another  limitation  was  the  availability  and  accessibility  of  each  of  the  experts.  As 
previously  mentioned,  each  of  the  experts  selected  were  all  senior  executive  IT  leaders 
within  their  organization.  Gaining  access  to  these  individuals  can  be  difficult  enough 
without  trying  to  convince  them  to  fill  out  three  questionnaires,  including  one  with  83 
questions  and  another  that  required  a  substantial  amoimt  of  writing  for  some.  Most  all  of 
the  selected  experts  were  very  helpful  and  accommodating.  However,  given  the  time 
constraints  coupled  with  a  change  of  administration,  conducting  another  round  would  not 
have  been  feasible. 

Also,  as  mentioned  in  chapter  four,  item  19,  was  deleted  because  during  the  final 
review  it  was  determined  that  the  wrong  question  had  been  attached  to  this  items  feedback 
statistics  in  the  second  and  third  round  surveys.  The  question  that  was  accidentally 
substituted  in  its  place  was  Increase  hardware  utilization,  which  had  a  true  average  of 
2.80.  The  original  question  was  Reassign  current  internal  IT  resources  to  higher  priority 
activities  and  had  the  advertised  average  of  4.20.  Therefore,  if  that  value  had  remained 
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constant,  this  item  would  have  been  included  in  the  final  top  ten  determinates.  However, 
this  error,  while  unfortunate,  lends  some  credibility  to  the  Delphi  process  employed  in  this 
study.  Closer  examination  of  the  results  shows  that  after  the  second  round  the  average  for 
this  question  had  dropped  to  3.71  and  a  ranking  of  17.  After  the  third  round  it  feU  even 
further  to  3.57  and  a  ranking  of  23.  This  trend  is  evidence  that  the  group  was  working  to 

lower  this  question  to  its  original  value  of  2.80. 

Another  potential  limitation  concerns  the  feedback  provided  to  the  group 
members.  As  previously  mentioned,  the  mean  response  of  each  item  in  the  previous 
questionnaire  was  used  as  feedback.  However,  this  statistic  assumes  normality  for  each 
items  histogram.  While  a  check  of  normality  was  performed  after  the  first  round  on  the 
total  number  of  each  response,  the  same  was  not  done  on  an  item  basis.  Therefore,  if  a 
bimodal  distribution  existed  for  a  particular  question  it  is  conceivable  that  any  outliers 
might  have  been  overly  weighted.  Also,  since  the  threshold  at  the  end  of  each  round  was 
appUed  to  the  mean  values,  it  is  possible  that  some  items  were  incorrectly  deleted  because 
of  the  potential  over  emphasis  on  outliers.  A  review  of  the  histograms  for  each  item  in 
each  round  did  show  a  bimodal  distribution  for  a  certain  percentage  of  the  items.  Table  20 
below  summarizes  these  findings. 


Table  20.  Percentage  of  Questions  with  Bimodal  Distributions 


Question  Set 

%  with  Bimodal  Distributions 

Round  1  -  All  Questions 

39.8% 

Round  1  -  Sub  set  (43  Questions) 

46.5% 

Round  2  -  All  Questions  (43  Questions) 

34% 

Round  2  -  Subset  (23  Questions) 

26% 

Round  3  -  All  Questions 

26% 
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As  observed  the  percentage  of  questions  with  a  bimodal  distribution  was  larger  in 
the  beginning.  However,  the  quick  decrease  in  this  number  was  expected  as  consensus 
started  to  buUd.  Also,  the  fact  that  the  final  23  questions  in  the  last  two  rounds  had  the 
same  percentage  is  perhaps  another  indicator  that  stability  had  been  achieved. 

Suggestions  for  Future  Research 

Now  that  an  investigation  has  been  made  as  to  why  the  DoD  outsources  its  IT, 
new  studies  can  use  it  as  a  foundation  to  bufid  upon.  One  possible  idea  for  fiiture  research 
iTiight  be  to  determine  if  these  determinants  are  the  best  set  for  the  DoD.  During  my 
interviews  it  was  painfully  obvious  that  many  senior  IT  executives  were  desperately  trying 
to  ascertain  the  best  set  of  reasons  to  outsource.  Now  that  a  determination  has  been  made 
as  to  what  outsourcing  determinants  are  currently  used,  a  study  could  be  undertaken  to 
determine  what  set  of  determinants  are  best.  Implicit  in  this  decision  is  an  evaluation  of 
whether  the  current  set  is  working  or  not.  One  possible  methodology  to  use  for  such  a 
study  might  be  to  analyze  eiqiectations  versus  actual  results.  Another  approach  might  be 
to  compare  figures  of  merit  before  and  after  outsourcing  such  as  cost,  service,  or  technical 
ability.  One  benefit  for  these  approaches  is  that  ten^lates  already  exist  (Lacity,  1994, 

Smith,  1998). 

Another  potential  research  question  might  be  to  determine  if  IT  outsourcing  should 
even  be  pursued  in  the  DoD  and  if  so,  how  much.  As  previously  stressed,  the  DoD  seems 
to  have  several  unique  organizational  requirements.  However,  as  this  research  suggests, 
the  differences  between  public  and  private  firms  seems  to  be  narrowing.  Therefore,  it 
might  be  beneficial  to  understand  if  these  differences  are  strong  enough  to  warrant 
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abandoning  outsourcing  and  re-ihtemalizing  all  IT  ftinctions  or  if  outsourcing  should  be 
considered  in  the  DoD  as  much  as  it  is  in  the  private  sector. 


Conclusions 

One  interesting  feature  of  this  research  is  that  it  reveals,  in  the  area  of  IT 
outsourcing,  the  DoD  seems  to  be  experiencing  much  the  same  evolution  the  private 
sector  did.  Currently,  it  seems  that  public  firms  seem  to  be  lagging  behind  many  pnvate 
organizations.  During  interviews  conducted  for  this  study,  their  were  many  who 
commented  that  when  it  came  to  IT  outsourcing,  the  DoD  was  about  three  too  four  years 
behind  the  private  sector  (Barclay,  2000;  Cooper,  2000).  This  realization  might  have  been 
motivation  for  inclusion  of  the  only  new  factor  suggested  by  the  group,  best  practices  and 
improved  processes  (NAl).  Perhaps  including  this  factor  is  a  realization  by  the  DoD  that 
they  have  something  to  learn  fi-om  private  firms  in  this  area.  Additionally,  even  though 
DoD  outsourcing  requirements  might  be  unique,  organizational  attitudes  and  behaviors 
towards  outsourcing  seems  to  be  following  the  same  evolution  they  did  in  the  pnvate 
sector.  There  seems  to  be  less  of  an  emphasis  now  being  placed  on  cost  related  factors 
and  more  being  placed  on  business  related  fectors.  Again,  this  seems  to  be  much  the  same 

evolution  that  private  sector  firms  eiqierienced. 

Another  thing  that  appears  obvious  is  that  each  organization  has  a  sUghtly  different 
focus  and  requirement  set.  Therefore,  each  of  their  uses  of  IT  outsourcing  will  be 
different.  This  is  consistent  with  previous  research  that  has  shown  that  population 
selection  for  any  IT  related  research  is  particularly  delicate  (Gefen,  1998).  Consequently, 
it  might  be  true  that  when  considering  IT  outsourcing,  it  is  most  important  to  look  within 
a  firm  to  determine  if  it  is  appropriate.  This  seems  to  be  consistent  with  a  new  wave  of 
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research  and  interest  in  this  area  that  focuses  on  right  or  smart  sourcing  (Lacity,  1996). 
This  research  stream  focuses  on  ensuring  that  a  firm  uses  IT  outsourcing  in  a  manner 
consistent  with  its  needs  and  requirements,  rather  than  applying  some  cookie  cutter 
approach  to  the  problem. 

With  less  emphasis  being  placed  on  the  military  in  the  national  agenda,  budgets  will 
probably  continue  to  fall.  However,  at  the  same  time,  it  has  almost  become  a  truism  that 
IT  in  any  organization  will  become  more  important.  Clearly,  a  process  to  lower  overall  IT 
costs  while  providing  the  required  level  of  information  services  to  DoD  members  is 
required.  Many  in  the  DoD  are  hoping  that  outsourcing  will  provide  the  solution. 
Hopefully,  this  research  has  provided  a  better  understanding  of  this  mode  of  IT 
governance  so  that  what  direction  and  goals  it  should  take  can  now  be  considered. 
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Appendix  A:  Questionnaire  #1  and  Cover  Letter 
This  appendix  contains  the  first  questionnaire  used  in  round  #1  and  the  accompanying 
cover  letter.  Please  note  that  the  numbering  scheme  used  in  this  survey  is  different  than  the  one 
presented  in  the  text.  To  match  the  two  schemes,  please  refer  to  Appendix  G  for  the  inter-round 
question  numbering  map. 
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DEPARTMENT  OF  THE  AIR  FORCE 

AIR  UNIVERSITY 

AIR  FORCE  INSTITUE  OF  TECHNOLOGY 


25  Sep  00 


Dear  study  participant 

Thank  you  very  much  for  contributing  to  this  research.  It  is  hoped  that  the  results  will  explain 
why  Department  of  Defense  agencies  outsource  their  information  technology  (IT)  systems.  Your 
support  helps  ensure  that  the  final  product  will  have  the  highest  reliability  and  validity  possible. 

This  study  is  based  on  the  Delphi  method.  The  Delphi  method  was  developed  by  the  RAND 
Corporation  and  is  a  formalized  process  to  determine  the  best  answer  or  solution  to  a  problem 
whenever  insufficient  or  no  applicable  data  exists.  It  uses  a  process  of  iterative  questionnaires  to 
help  form  consensus  fi"om  a  panel  of  experts.  After  each  round,  the  statistical  results  of  the  last 
questionnaire  are  presented  to  the  group,  along  with  another  shorter  questionnaire.  It’s  been 
shown  that  as  this  process  continues,  individual  results  will  converge  into  a  final  group  solution. 
The  first  questionnaire  should  take  approximately  20  minutes  to  complete,  the  next  15  minutes 
and  the  last  one  10  minutes.  You  can  expect  each  questionnaire  to  arrive  four  weeks  after  the 
previous  one. 

To  protect  your  privacy,  the  integrity  of  the  process,  and  allow  you  the  fi-eedom  to  provide 
unbiased  feedback,  your  name,  organization,  and  responses  will  be  kept  strictly  confidential.  If 
any  specific  responses  you  provided  need  to  be  published,  your  identity  wiU  be  conpletely 
masked. 

Again,  thank  you  very  much  for  you  input.  If  you  have  any  questions  please  feel  firee  to 
contact  me  at  (937)235-9946  or  via  email  at  alexander.barelka@,afit.afmil 

Sincerely 


Alex  J.  Barelka,  Capt,  USAF 
Graduate  Student,  AFIT 


Attachment: 

1.  Questionnaire 

2.  Return  envelope 


Information  Technology  (IT)  outsourcing  — 
Questionnaire  #1 


Niunerous  studies  have  considered  IT  outsourcing  in  the  private  sector,  but  few  have 
focused  on  the  Department  of  Defense  (DoD).  Consequently,  there  is  no  standard 
framework  for  why  IT  outsourcing  decisions  are  made  by  DoD  agencies.  Without  this 
information  it  is  difficult  to  make  any  predictions  about  the  outcome  of  future  outsourcing 
decisions. 

This  study  attempts  to  create  a  validated  list  of  determinants  for  why  DoD  agencies 
outsource  their  IT  systems.  By  providing  a  comprehensive  framework  for  why 
outsourcing  decisions  are  initially  made,  it  is  hoped  that  future  decisions  can  be  made  with 

less  imcertainty. 

By  completing  the  following  survey,  you  will  be  helping  to  develop  this  framework.  If 
there  are  any  factors  not  present  in  the  survey  which  you  think  are  important  to  an  IT 
outsourcing  decision,  please  write  them  down  in  the  spaces  provided  on  pages  6  and  10. 

Please  remember  your  name,  organization,  and  responses  will  be  kept  anonymous.  The 
specific  information  requested  below  is  only  for  internal  tracking  purposes.  Prompt  return 
of  the  survey  will  help  maintain  the  integrity  of  this  study.  Therefore,  please  try  your  best 
to  return  the  completed  questionnaire  via  fax  no  later  than  seven  calendar  days  after 
receipt.  The  dedicated  fax  number  for  the  study  is  (208)  575-6528.  You  may  also  mail 
the  survey  back  in  the  self-addressed  envelope  provided. 

If  you  have  any  questions  regarding  this  effort  please  don’t  hesitate  to  contact  me  at  (937) 
235-9946  (h),  or  via  e-mail  at  alexander.barelka@afit.af.mil.  Thank  you  again  for  your 
time  and  assistance. 
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General  Information:  Please  print  legibly 
Name:  _ _ _ 

Grade/Rank: : _ 

Current  organization: _ 

Current  position: _ _ _ 

Months  in  current  position: _ 

Mmling  Address:^ _ _ _ 


Fax  number: _ 

Primary  duties: 


Have  you  ever 


been  a  member  of  an  outsourcing  event?  If  so,  v^hat  role  did  you  play? 


Listed  below  are  several  potential  reasons  why  an  organization  might  or  might  not 
outsource  its  IT  systems.  Please  indicate  the  amovmt  of  consideration  you  feel  is 
appropriate  for  each  reason  when  making  an  IT  outsourcing  decision  for  your  agency.  The 
scale  ranges  from  a  low  of  1  to  a  high  of  5.  Please  note  the  question  is  whether  you 
consider  these  factors  now,  not  if  you  should  in  the  future. 


Reason 

Amount  of  Consideration 

Low 

High 

1 .  Lower  cost  due  to  a  vendors  economies  of  scale 

1  2 

3 

4 

5 

2.  Ability  of  IT  outsourcing  to  simplify  management 
agenda  (decreases  management  workload) 

1  2 

3 

4 

5 

3.  Improve  technical  services  (e.g.  dissatisfaction  with 
current,  in-house  IT  support) 

1  2 

3 

4 

5 

4.  High  confidence  that  the  contractor  will  perform  to 
government  expectations 

1  2 

3 

4 

5 

5.  Lower  costs  due  to  a  vendor’s  tighter  control  over 
fringe  benefits 

1  2 

3 

4 

5 

6.  Ability  of  IT  outsourcing  to  flatten  organizations 

1  2 

3 

4 

5 

7.  Increase  customer  satisfaction  of  IT  related  services 

1  2 

3 

4 

5 

8.  Access  to  a  vendor’s  large  research  and  development 
efforts 

1  2 

3 

4 

5 

9.  Lower  costs  by  vendors  relocating  data  centers  to 
cheaper  areas 

1  2 

3 

4 

5 

10.  Ability  of  IT  outsourcing  to  leverage  innovation 

1  2 

3 

4 

5 

1 1 .  Lower  costs  due  to  a  vendor’s  more  focused  expertise 
in  managing  IT 

1  2 

3 

4 

5 

12.  Improve  the  long-term  financial  predictability  of  IT 
budgets 

■ 

3 

4 

5 

123 


Amount  of  Consideration 


13.  Increase  employee  motivation  and  cohesion 


14.  Improve  cost  controls  (e.g.  develop  standardized 
processes  for  all  agency  members  to  follow) 


15.  Act  as  a  change  agent  for  reorganization 


16.  Demonstrate  compliance  with  government  guidelines 


17.  Absorb  unnecessary  IT  employees  generated  by  agency 
merger 


18.  Convert  large  IT  capital  budgets  into  more  flexible 
operating  budgets 


19.  Increase  hardware  utilization 


20.  Comply  with  Government  requirements  to  outsource 


21.  Reassign  current  internal  IT  resources  to  higher 
priority  activities 


22.  Absorb  excess  IT  assets  after  reorganization 


23.  Improve  quantity  or  quality  of  agency  outputs 


24.  Solve  technical  incompatibilities  after  reorganization 


25.  Potential  for  hidden  costs  (e.g.  imejqpected  costs  for 
assumed  IT  services) 


26.  Productivity  improvements 


27.  Retain  competent  IT  professionals 


28.  Inability  of  a  vendor  to  guarantee  interoperability  with 
system  they  don’t  control 


29.  Inability  of  the  vendor  to  meet  system  availability 
requirements 


2 

3  4 

2 

3  4 

2 

3  4 

2 

3  4 

2 

3  4 

2 

3  4 

2 

3  4 

2 

3  4 

2 

3  4 

1  2 

3  4 

1  2 

3  4 

1  2 

3  4 

1  2 

3  4 

1  2 

3  4 

1  2 

3  4 

1  2 

3  4 

1  2 

3  4 
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Amount  of  Consideration 


30.  Vendor  not  upgrading  or  keeping  staff  proficient 


31.  Complications  between  vendor  and  customer  technical 
IT  interfaces 


32.  Restructuring  IT  budgets  to  allow  dijfferent 
appropriations  to  be  used 


33.  Protect  and  stabilize  IT  budgets  (e.g.  placing  IT  on 
contract  to  protect  funding) 

34.  Potential  that  poor  performing  agency  personnel, 
whose  jobs  have  been  outsourced,  will  be  enployed  by 
the  vendor 

35.  Large  number  of  integration  and  system  dependencies 


36.  Security  concerns  over  granting  an  IT  vendor’s 
maintenance  personnel  access  to  a  DoD  IT  systems 


37.  Increased  transactional  costs  (contract  processing, 
administration,  coordination,  etc.) 

38.  Feelings  that  vendor  vdll  not  be  able  to  solve  agency 
specific  IT  requirements 


39.  Record  all  costs  incurred  by  the  users 


40.  Locate  competent  IT  professionals 


41 .  Lose  control  over  IT  services 


42.  Security  concerns  over  allowing  outside  security 
specialists  access  to  IT  systems 


43.  Security  concerns  over  how  vendors  handle  agency 
data  on  their  own  systems 


44.  Inability  to  meet  recovery  and  back-up  requirements 


45.  Potential  release  of  system  design  information 


Amount  of  Consideration 

46.  Loss  of  critical  IT  skills  base 

1 

2 

3 

4 

5 

47.  Concerns  over  a  vendors  e-mail  security  procedures 
when  discussing  agency  issues 

1 

2 

3 

4 

5 

48.  Potential  for  inflated  cost  savings  projections 

1 

2 

3 

4 

5 

49.  Becoming  overly  dependent  on  contractors 

1 

2 

3 

4 

5 

50.  Security  concerns  about  subcontractors 

1 

2 

3 

4 

5 

5 1 .  Concerns  over  how  a  vendor  disposes  agency 
information  after  it  is  no  longer  needed 

1 

2 

3 

4 

5 

52.  Lack  of  trust  between  Government  and  contractors 

1 

2 

3 

4 

5 

53.  Loss  of  control  over  timing  and  quality  of  outputs 

1 

2 

3 

4 

5 

54.  Security  risk  that  technical  design  information  will  be 
compromised 

1 

2 

3 

4 

5 

55.  Possibility  that  vendor  will  disclose  classified  data 

1 

2 

3 

4 

5 

Please  list  and  rate  any  other  important  reasons  why  you  think  an 
not  outsource  its  IT  systems. 

agency  might  or  might 

Level  of  Comfort 

Statement 

Low 

High 

1 

2 

3 

4 

5 

1 

2 

3 

4 

5 
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Level  of  Comfort 

1  2  3  4  5 

Listed  below  are  several  statements  related  to  your  agency  and  IT  outsourcing.  Please 
rate  your  level  of  agreement  for  each  statement.  The  scale  ranges  from  a  low  of  1  to  a 
high  of  5. 


Level  of  Agreement 

Statement 

Low 

High 

1 .  I  believe  my  agency  has  a  high  abOity  to  wnte  an 
adequate  IT  service  level  agreement 

1 

2 

3 

4 

5 

2.  I  believe  my  response  to  the  question  above  impacts 
my  decision  to  outsource 

1 

2 

3 

4 

5 

3 .  I  believe  my  agency  has  the  ability  to  properly  manage 
an  IT  contract 

1 

2 

3 

4 

5 

4.  I  believe  my  response  to  the  question  above  impacts 
my  decision  to  outsource 

1 

2 

3 

4 

5 

5.  I  believe  IT  outsourcing  lowers  my  agency’s  ability  to 

maximize  its  business  opportunities 

1 

2 

3 

4 

5 

6.  I  believe  the  length  of  time  an  IT  manager  knows  they 
will  be  in  a  position  affects  their  decision  to  outsource 

1 

2 

3 

4 

5 

7.  I  believe  concern  over  displaced  IT  staff  members 
affects  my  decision  to  outsource 

1 

2 

3 

4 

5 

8.  I  believe  my  agency  is  capable  of  establishing  its 
information  requirements 

1 

2 

3 

4 

5 

9.  I  believe  IT  contracts  can  be  adequately  managed  by 
my  agency’s  current  personnel 

1 

2 

3 

4 

5 
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Level  of  Agreement 

10. 1  believe  IT  outsourcing  decreases  my  agency’s  ability 
to  fiiUy  leverage  IT 

1  2  3  4  5 

1 1 . 1  believe  the  longer  an  IT  manager  is  in  a  position,  the 
more  it  will  affect  their  decision  to  outsource 

1  2  3  4  5 

12. 1  believe  decisions  to  start  outsourcing  are  influenced 
by  peoples  perceptions  on  how  it  affects  them 

1  2  3  4  5 

13.1  believe  my  agency  can  adequately  establish  contract 
deliverables  from  its  information  requirements 

1  2  3  4  5 

14. 1  believe  my  response  to  the  question  above  impacts 
my  decision  to  outsource 

1  2  3  4  5 

15. 1  believe  some  IT  managers  solicit  outsourcing 

proposals  in  the  hopes  it  will  prove  the  efficiency  and 
continued  existence  of  the  internal  IT  department. 

1  2  3  4  5 

16. 1  believe  some  IT  managers  solicit  outsourcing 
proposals  so  that  they  can  get  senior  leaders  to 
authorize  additional  funding  for  an  IT  program  by 
showing  them  outsourcing  is  not  more  cost  effective 

1  2  3  4  5 

17. 1  believe  some  IT  managers  consider  outsourcing  to 
expose  exaggerated  claims  made  by  vendors,  thereby 
working  to  ensure  senior  leaders  do  not  enter  an 
outsourcing  relationship  prematurely 

1  2  3  4  5 

18.1  believe  some  IT  managers  outsource  to  eliminate  a 
troublesome  function  that  is  “not  worth  the  headaches" 

1  2  3  4  5 

19. 1  believe  some  IT  managers  outsource  to  appear  more 
corporate  and  credible  in  the  hopes  of  enhancing  their 
own  career 

1  2  3  4  5 

20. 1  believe  my  agency  can  develop  a  proper  acquisition 
strategy  for  its  information  requirements 

1  2  3  4  5 

21.1  believe  my  response  to  the  question  above  impacts 
my  decision  to  outsource 

1  2  3  4  5 
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Level  of  Agreement 

22. 1  believe  some  IT  managers  attempt  to  convince  senior 
managers  that  their  internal  IT  departments  can 
perform  the  same  services  an  outside  vendor  can 
perform  for  the  same  amount  or  less 

1  2 

3 

4 

5 

23. 1  believe  some  IT  managers  justify  new  resources  by 
starting  an  IT  source  selection  knowing  that  vendors 
will  not  be  competitive  against  the  internal  department 

1  2 

3 

4 

5 

24. 1  believe  some  IT  managers  will  start  outsourcing 
evaluations  to  prove  to  senior  leadership  that 
outsourcing  might  not  meet  their  expectations 

1  2 

3 

4 

5 

25. 1  believe  some  IT  managers  outsource  IT  to  simply 
eliminate  a  problematic  function 

I  2 

3 

4 

5 

26. 1  believe  the  more  other  government  agencies 
outsource  their  IT,  the  more  mine  might 

I  2 

3 

4 

5 

27. 1  believe  some  IT  managers  outsource  in  the  hopes  that 
senior  management  will  appreciate  and  value  them 
more 

1  2 

3 

4 

5 

28. 1  believe  some  IT  managers  are  interested  in  IT 
outsourcing  only  because  other  IT  managers  in  the 
government  are 

1  2 

3 

4 

5 
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Please  list  and  rate  any  other  statements  you  feel  are  important  to  IT  outsourcing. 


Please  list  any  additional  comments  you  have  about  this  survey  or  its  subject  matter: 


Thank  you  for  your  time 


3 


Appendix  B:  Questionnaire  #2  and  Cover  Letter 

This  appendix  contains  the  first  questionnaire  used  in  round  #2  and  the  accompanying 
cover  letter.  Please  note  that  the  numbering  scheme  used  in  this  survey  is  different  than 
the  one  presented  in  the  text.  To  match  the  two  schemes,  please  refer  to  Appendix  G  for 
the  inter-round  question  numbering  map. 
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DEPARTMENT  OF  THE  AIR  FORCE 

AIR  UNIVERSITY 

AIR  FORCE  INSTITUE  OF  TECHNOLOGY 


29  Nov  00 


Dear  study  participant 

Thank  you  very  much  for  your  responses  to  the  first  round  of  this  study.  The  input  has  been 
great  and  so  far  the  results  look  very  interesting.  Based  on  the  scores  provided  in  the  l^t  round 
by  the  group,  each  question  that  had  an  average  value  below  3.0  h^  been  dropped.  This 
represents  a  total  reduction  of  about  50%.  However,  due  to  technical  difficulties,  another  table, 
with  ten  new  questions  needed  to  be  included. 

During  the  last  round,  some  study  participants  correctly  pointed  out  that  the  instructions  did 
not  specify  what  type  ofoutsourcing  is  being  examined.  This  was  an  oversight.  This  research 
primarily  confines  itself  to  the  examination  of  operations  and  maintenance  related  functions  and 
high  dollar  value  contracts  such  as  large  support  and  services  agreements.  Exanqiles  of  items  that 
would  not  be  considered  in  such  a  definition  are  software  or  hardware  development,  or  IT 
research. 

Some  have  also  expressed  an  opinion  that  the  first  questionnaire  did  not  flow  well.  was 
anticipated  due  to  the  adherence  to  a  strict  interpretation  of  survey  admimstration  guidelines. 
However,  these  guidelines  have  since  been  relaxed,  so  you  will  hopefully  find  this  version  easier 
to  follow!  I  have  also  attempted  to  format  the  survey  in  an  easy  to  read  layout.  However,  I  have 
noticed  that  for  whatever  reason,  some  printers  do  not  faithfully  reproduce  the  intended  product. 

Again,  thank  you  very  much  for  you  input.  If  you  have  any  questions  please  feel  fi’ee  to 
contact  me  at  (937)  235-9946  or  via  email  at  alexander.barelka@afit.af..rnil 

Sincerely 


Alex  J.  Barelka,  Capt,  USAF 
Graduate  Student,  AFIT 


Attachment: 

1.  Questionnaire 
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Information  Technology  (IT)  outsourcing  - 
Questionnaire  #2 

Participant  Name: 


As  you  know,  this  study  is  attempting  to  create  a  validated  list  of  determinants  for  why 
DoD  agencies  outsource  their  IT  systems.  By  developing  a  comprehensive  framework  for 
why  outsourcing  decisions  are  initiaUy  made,  it  is  hoped  that  future  decisions  can  be  made 
with  less  uncertainty.  Wlule  the  purpose  of  the  first  round  was  to  identify  the  top  rated 
determinants,  the  purpose  of  this  second  round  is  to  provide  you  with  feedback  and  to 
further  define  the  list. 

Please  notice  that  next  to  each  of  the  following  questions  is  your  previous  score  and  the 
group  average.  Please  consider  this  information  and  re-score  each  item  as  you  see  fit. 
Also,  for  those  questions  which  indicate  that  your  previous  response  fell  more  than  1.25 
standard  deviations  away  from  the  mean,  please  comment  on  why  you  think  this  might 
have  occurred  in  the  space  provided. 

Also,  several  additional  factors  provided  by  group  members  have  also  been  incoiporated. 
Again  if  there  are  still  any  fectors  not  present  in  the  survey  which  you  think  are  important 
to  an  IT  outsourcing  decision,  please  write  them  down  in  the  spaces  provided. 

Prompt  return  of  the  survey  will  help  maintain  the  integrity  of  this  study.  Therefore, 
please  try  your  best  to  return  the  completed  questionnaire  via  fax  no  later  than  seven 
calendar  days  after  receipt.  The  dedicated  fax  number  for  the  study  is  (208)  575-^28. 
You  may  also  e-mail  your  reply  after  simply  highlighting  your  selections  with  a  different 

color.  Or,  you  can  mail  the  survey  to: 

AHT/ENV  BLDG  640 
2950  P  STREET 

WRIGHT-PATTTERSON  AFB  OH  45433-7765 

If  you  have  any  questions  regarding  this  effort  please  don’t  hesitate  to  contact  me  at  (937) 
235-9946  (h),  or  via  e-mail  at  alexander.barelka@afit.afmil.  Thank  you  again  for  your 
time  and  assistance. 
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Listed  below  are  several  potential  reasons  why  an  organization  might  or  might  not 
outsource  its  IT  systems.  Please  indicate  the  amount  of  consideration  you  feel  is 
appropriate  for  each  reason  when  making  an  IT  outsourcing  decision  for  your  agency.  The 
scale  ranges  from  a  low  of  1  to  a  high  of  5.  Please  note  the  question  is  whether  you 
consider  these  fectors  now,  not  if  you  should  in  the  future. 


Group 

Average 

Your 

Last 

Answer 

Reason 

Amount  of  Consideration 

Low 

High 

3.8 

1 .  Lower  cost  due  to  a  vendors  economies  of 
scale 

1 

2 

3 

4 

5 

3.8 

2.  Lower  costs  due  to  a  vendor’s  more  focused 
ejq)ertise  in  managing  IT 

1 

2 

3 

4 

5 

4 

3.  Improve  cost  controls  (e.g.  develop 
standardized  processes  for  all  agency 
members  to  follow) 

1 

2 

3 

4 

5 

3 

4.  Record  all  costs  incmred  by  the  users 

1 

2 

3 

4 

5 

3.1 

5.  Convert  large  IT  capital  budgets  into  more 
flexible  operating  budgets 

1 

2 

3 

4 

5 

3.4 

6.  Improve  the  long-term  financial 
predictability  of  IT  budgets 

1 

2 

3 

4 

5 

3.200 

7.  Protect  and  stabilize  IT  budgets  (e.g.  placing 
IT  on  contract  to  protect  funding) 

1 

2 

3 

4 

5 

3.8 

8.  Ability  of  IT  outsourcing  to  simplify 

management  agenda  (decreases  management 
workload) 

1 

2 

3 

4 

5 

3.8 

9.  Ability  of  IT  outsourcing  to  leverage 
innovation 

1 

2 

3 

4 

5 

4.2 

10.  Increase  hardware  utilization 

1 

2 

3 

4 

5 
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Amount  of  Consideration 

4.3 

1 1 .  Improve  quantity  or  quality  of  agency 
outputs 

1 

2 

3 

4 

5 

4.5 

12.  Productivity  improvements 

1 

2 

3 

4 

5 

3.2 

13.  Act  as  a  change  agent  for  reorganization 

1 

2 

3 

4 

5 

4.1 

14.  Improve  technical  services  (e.g. 

dissatisfaction  with  current,  in-house  IT 
support) 

1 

2 

3 

4 

5 

4 

15.  Increase  customer  satisfaction  of  IT  related 
services 

1 

2 

3 

4 

5 

3.125 

16.  Retain  competent  IT  professionals 

1 

2 

3 

4 

5 

4.111 

17.  Locate  competent  IT  professionals 

1 

2 

3 

4 

5 

3.1 

18.  Access  to  a  vendor’s  large  research  and 
development  efforts 

1 

2 

3 

4 

5 

3.125 

19.  Potential  for  hidden  costs  (e.g.  unejqjected 
costs  for  assumed  IT  services) 

1 

2 

3 

4 

5 

3.222 

20.  Increased  transactional  costs  (contract 
processing,  administration,  coordination, 
etc.) 

1 

2 

3 

4 

5 

3.333 

21.  Lose  control  over  IT  services 

1 

2 

3 

4 

5 

3.778 

22.  Loss  of  critical  IT  skills  base 

1 

2 

3 

4 

5 

3 

23.  Becoming  overly  dependent  on  contractors 

1 

2 

3 

4 

5 

3 

24.  Loss  of  control  over  timing  and  quality  of 
outputs 

1 

2 

3 

4 

5 
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Amount  of  Consideration 

3.111 

25.  Feelings  that  vendor  will  not  be  able  to  solve 
agency  specific  IT  requirements 

1  2 

3 

4 

5 

3.2 

26.  Complications  between  vendor  and  customer 
technical  IT  interfaces 

1  2 

3 

4 

5 

3.444 

27.  Security  concerns  over  granting  an  IT 

vendor’s  maintenance  personnel  access  to  a 
DoD  IT  systems 

1  2 

3 

4 

5 

3.4 

- 

28.  Security  concerns  over  allovmg  outside 
security  specialists  access  to  IT  systems 

1  2 

3 

4 

5 

3 

29.  Security  concerns  about  subcontractors 

1  2 

3 

4 

5 

3.375 

30.  Possibility  that  vendor  will  disclose  classified 
data 

1  2 

3 

4 

5 

4 

31 .  Security  concerns  over  how  vendors  handle 
agency  data  on  their  own  systems 

1  2 

3 

4 

5 

3 

32.  Concerns  over  a  vendors  e-mail  security 
procedures  when  discussing  agency  issues 

1  2 

3 

4 

5 

3 

33.  Concerns  over  how  a  vendor  disposes 
agency  information  after  it  is  no  longer 
needed 

1  2 

3 

4 

5 

3.1 

34.  Inability  of  a  vendor  to  guarantee 
interoperability  with  system  they  don’t 
control 

1  2 

3 

4 

5 

3.889 

35.  Large  number  of  integration  and  system 
dependencies 

1  2 

3 

4 

5 

Below  ai 

•e  new  que 

_i - - - - - - - - - - - 

stions  submitted  by  group  members  during  the  last  round 

36.  Congressional  Interest 

1  2 

3 

4 

5 
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Amount  of  Consideration 

37.  Great 

L - — - - - - - - 

e  agency  wide  business  practice 

1  2 

3 

4  5 

38.  Ability  of  IT  outsourcing  to  make  interoperability  between 
systems  LANs  better 

1  2 

3 

4  5 

39.  Quality  of  Intra-agency  relationships  (between  IT  and  business 
units) 

1  2 

3 

4  5 

40.  Lack  of  a  clearly  delineated  set  of  systems,  costs  and  service  levels 

1  2 

3 

4  5 

41 .  Controlling  different  security  levels  of  data  without  security 
incident 

1  2 

3 

4  5 

Please  list  and  rate  any  other  important  reasons  why  you  think  an  agency  might  or  might 
not  outsource  its  IT  systems. 


Statement 

Level  of  Comfort 

Low  High 

1  2  3  4  5 

1  2  3  4  5 

1  2  3  4  5 
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Listed  below  are  several  statements  related  to  your  agency  and  IT  outsourcing.  Please 
rate  your  level  of  agreement  for  each  statement.  The  scale  ranges  from  a  low  of  1  to  a 

high  of  5. 


Group 

Your 

Last 

Answer 

Statement 

Level  of  Agreement 

Average 

Low 

High 

3.7 

1 .  I  believe  some  IT  managers  attempt  to 
convince  senior  managers  that  their  internal 

IT  departments  can  perform  the  same 
services  an  outside  vendor  can  perform  for 
the  same  amount  or  less 

1 

2 

3 

4 

5 

3.6 

2.  I  believe  some  IT  managers  outsource  to 
eliminate  a  troublesome  function  that  is  “not 
worth  the  headaches" 

1 

2 

3 

4 

5 

3.8 

3 .  I  believe  some  IT  managers  outsource  IT  to 
simply  eliminate  a  problematic  function 

1 

2 

3 

4 

5 

3 

4.  I  believe  the  length  of  time  an  IT  manager 
knows  they  will  be  in  a  position  affects  their 
decision  to  outsource 

1 

2 

3 

4 

5 

3.3 

5.  I  believe  the  longer  an  IT  manager  is  in  a 
position,  the  more  it  will  affect  their  decision 
to  outsource 

1 

2 

3 

4 

5 

4.2 

6.  I  believe  the  more  other  government 

agencies  outsource  their  IT,  the  more  mine 
might 

1 

2 

3 

4 

5 

3.2 

7.  I  believe  some  IT  managers  are  interested  in 
IT  outsourcing  only  because  other  IT 
managers  in  the  government  are 

1 

2 

3 

4 

5 

3.6 

8.  I  believe  decisions  to  start  outsourcing  are 
influenced  by  peoples  perceptions  on  how  it 
affects  them 

J _ _ _ _ _ 

1 

2 

3 

4 

5 
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Level  of  Agreement 

Below  an 

new  questions  submitted  by  group  members  during  the  last  round 

9.  I  believe  that  many  agencies  feel  that  outsomcing  is  either 

mandated  or  so  strongly  encouraged  as  to  be,  in  essence  mandated 

1 

2 

3 

4 

5 

10. 1  believe  that  many  senior  IT  managers  see  outsourcing  as  an  end  in 
itself  rather  than  a  strategy  to  achieve  a  goal 

1 

2 

3 

4 

5 

1 1. 1  believe  outsourcing  activities  are  not  initiated  by  IT  organizations. 
External  actors  loom  large  in  decisions 

1 

2 

3 

4 

5 

12. 1  believe  some  IT  managers  view  outsourcing  as  a  means  to  adopt 
industry  best  practices  and  improve  processes 

1 

2 

3 

4 

5 

13.1  believe  some  IT  managers  view  outsourcing  as  a  means  to 
mitigate  the  inability  of  government  to  remove  incompetent  IT 
workers 

1 

2 

3 

4 

5 

14. 1  believe  the  cost  of  outsourcing  is  not  clearly  understood 

1 

2 

3 

4 

5 

15. 1  believe  outsourcing  should  not  be  done  without  a  comprehensive 
business  assessment  to  establish  value 

1 

2 

3 

4 

5 

16. 1  believe  managers  do  not  determine  best  value  prior  to  outsourcing 

1 

2 

3 

4 

5 

Please  list  and  rate  any  other  statements  you  feel  are  important  to  IT  outsourcing. 


Statement 

Level  of  Comfort 

Low  High 

1  2  3  4  5 
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Level  of  Comfort 

1  2  3  4  5 

1  2  3  4  5 

Below  are  similar  questions  from  the  first  round  that  needed  to  be  re-worded  to  better 
comply  with  survey  administration  procedures. 


Statement 

Level  of  Agreement 

Low 

High 

1 .  I  believe  my  agency  has  a  low  ability  to  write  an 
adequate  IT  service  level  agreement 

1 

2 

3 

4 

5 

2.  I  believe  my  response  to  the  question  above  inpacts 
my  decision  to  outsource 

1 

2 

3 

4 

5 

3.  I  believe  my  agency  does  not  have  the  ability  to 
properly  manage  an  IT  contract 

1 

2 

3 

4 

5 

4.  I  believe  my  response  to  the  question  above  inqiacts 
my  decision  to  outsource 

1 

2 

3 

4 

5 

5.  I  believe  my  agency  is  not  capable  of  establishing  its 
information  requirements 

1 

2 

3 

4 

5 

6.  I  believe  IT  contracts  can  not  be  adequately  managed 
by  my  agency’s  current  personnel 

1 

2 

3 

4 

5 

7.  I  believe  my  agency  can  not  adequately  establish 
contract  deliverables  from  its  information 
requirements 

1 

2 

3 

4 

5 
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Level  of  Agreement 

8.  I  believe  my  response  to  the  question  above  impacts 
my  decision  to  outsource 

1  2  3  4  5 

9.  I  believe  my  agency  can  not  develop  a  proper 

acquisition  strategy  for  its  information  requirements 

1  2  3  4  5 

10. 1  believe  my  response  to  the  question  above  in^acts 
my  decision  to  outsource 

1  2  3  4  5 

Please  list  any  additional  comments  you  have  about  this  survey  or  its  subject  matter: 


Thank  you  for  your  time. . . 
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Appendix  C:  Questionnaire  #3  and  Cover  Letter 
This  appendix  contains  the  first  questionnaire  used  in  round  #3  and  the  accompanying 
cover  letter.  Please  note  that  the  numbering  scheme  used  in  this  survey  is  different  than 
the  one  presented  in  the  text.  To  match  the  two  schemes,  please  refer  to  Appendix  G  for 


the  inter-round  question  numbering  map. 


DEPARTMENT  OF  THE  AIR  FORCE 

AIR  UNIVERSITY 

AIR  FORCE  INSTITUE  OF  TECHNOLOGY 


11  Jan  01 


Dear  study  participant 

Let  me  begin  by  again  thanking  everyone  for  their  continued  support  dumg  this  study.  I 
realize  that  all  of  you  have  very  important  matters  to  attend  to  and  that  finding  time  to  fill  out 
these  questionnaire  was  sometimes  difficult.  However,  your  efforts  do  not  appear  to  have  been  m 
vain.  So  far  the  results  look  very  interesting  and  will  hopefully  provide  direct  and  immediate 
benefits  to  all  DoD  CIO  offices. 

Once  the  study  is  concluded,  I  will  be  developing  an  executive  overview  of  the  results  for 
your  review.  If  you  would  like,  I  can  personally  brief  them  to  you  or  your  staff  members.  I 
be  moving  back  to  the  Washington  D.C.  area  in  April  so  such  a  meeting  should  not  be  dfficull  to 
arrange.  If  you  want  the  in-person  briefing  simply  annotate  that  on  the  questionnaire,  otherwise,  I 

will  sinqily  send  you  the  material  via  e-mail. 

As  promised  this  survey  is  shorter;  only  23  questions,  no  writing  required.  Based  on  the 
scores  provided  in  the  last  round  by  the  group,  each  question  that  had  an  average  value  below 
3.714  has  been  dropped.  This  represents  another  reduction  of  about  50%. 

Again,  thank  you  very  much  for  you  input.  If  you  have  any  questions  please  feel  firee  to 
contact  me  at  (937)  235-9946  or  via  email  at  alexander.barelka(a), afit.af.mil 


Sincerely 


Alex  J.  Barelka,  Capt,  USAF 
Graduate  Student,  AFIT 


Attachment: 

1.  Questionnaire 
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Information  Technology  (IT)  outsourcing  - 
Questionnaire  #3 


Participant  Name: 

As  you  know,  this  study  is  attempting  to  create  a  validated  list  of  determinants  for  why 
DoD  agencies  outsource  their  IT  systems.  By  developing  a  con^rehensive  framework  for 
why  outsourcing  decisions  are  initially  made,  it  is  hoped  that  future  decisions  can  be  nmde 
with  less  uncertainty.  While  the  purpose  of  the  &st  round  was  to  identify  the  top  rated 
determinants,  the  purpose  of  this  second  round  is  to  provide  you  with  feedback  and  to 
further  define  the  Ust. 

Prompt  return  of  the  survey  will  help  maintain  the  integrity  of  this  study.  Therefore, 
please  try  your  best  to  return  the  completed  questionnaire  via  fax  no  later  than  seven 
calendar  days  after  receipt.  The  dedicated  fax  number  for  the  study  is  (208)  575-6528. 
You  may  also  e-mail  your  reply  after  simply  highlighting  your  selections  with  a  different 
color.  Or,  you  can  mail  the  survey  to: 

AFIT/ENV  BLDG  640 
2950  P  STREET 

WRIGHT-PATTTERSON  AFB  OH  45433-7765 

If  you  have  any  questions  regarding  this  effort  please  don’t  hesitate  to  contact  me  at  (937) 
235-9946  (h),  or  via  e-mail  at  alexander.barelka@nfit.af.mil.  Thank  you  again  for  your 
time  and  assistance. 
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Listed  below  are  several  potential  reasons  why  an  organization  might  or  might  not 
outsource  its  IT  systems.  Please  indicate  the  amount  of  consideration  you  feel  is 
appropriate  for  each  reason  when  making  an  IT  outsourcing  decision  for  your  agency.  The 
scale  ranges  from  a  low  of  1  to  a  high  of  5.  Please  note  the  question  is  whether  you 
consider  these  factors  now,  not  if  you  should  in  the  future. 


Group 

Average 

Your 

Last 

Answer 

Reason 

Amount  of  Consideration 

Low 

High 

3.71 

1. 

Lower  cost  due  to  a  vendors  economies  of 
scale 

1 

2 

3 

4 

5 

3.71 

2. 

Lower  costs  due  to  a  vendor’s  more 
focused  expertise  in  managing  IT 

1 

2 

3 

4 

5 

3.71 

3. 

Improve  cost  controls  (e.g.  develop 
standardized  processes  for  all  agency 
members  to  follow) 

1 

2 

3 

4 

5 

3.71 

4. 

Increase  hardware  utilization 

2 

3 

4 

5 

4.29 

5. 

Improve  quantity  or  quality  of  agency 
outputs 

2 

3 

4 

5 

4.43 

6. 

Productivity  improvements 

2 

3 

4 

5 

4.14 

7. 

Inq)rove  technical  services  (e.g. 
dissatisfaction  with  current,  in-house  IT 
support) 

1 

2 

3 

4 

5 

3.86 

8. 

Increase  customer  satisfaction  of  IT 
related  services 

1 

2 

3 

4 

5 

3.86 

9. 

Locate  competent  IT  professionals 

1 

2 

3 

4 

5 

3.86 

10.  Security  concerns  over  how  vendors 
handle  agency  data  on  their  own  systems 

1 

2 

3 

4 

5 
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Amount  of  Consideration 

4.14 

1 1 .  Create  agency  vvade  business  practice 

1  2  3  4  5 

4.00 

12.  Lack  of  a  clearly  delineated  set  of  systems, 
costs  and  service  levels 

1  2  3  4  5 

3.86 

13.  Controlling  different  security  levels  of  data 
without  security  incident 

1  2  3  4  5 

Listed  below  are  several  statements  related  to  your  agency  and  IT  outsourcing.  Please 
rate  your  level  of  agreement  for  each  statement.  The  scale  ranges  from  a  low  of  1  to  a 
high  of  5. 


Group 

Average 

Your 

Last 

Answer 

Statement 

Level  of  Agreement 

Low  High 

4.14 

1 .  I  believe  some  IT  managers  attempt  to 
convince  senior  managers  that  their 
internal  IT  departments  can  perform  the 
same  services  an  outside  vendor  can 
perform  for  the  same  amount  or  less 

1  2  3  4  5 

4.14 

2.  I  believe  some  IT  managers  outsource  to 
eliminate  a  troublesome  function  that  is 
“not  worth  the  headaches" 

1  2  3  4  5 

3.86 

3.  I  believe  some  IT  managers  outsource  IT 
to  simply  eliminate  a  problematic  ftinction 

1  2  3  4  5 

3.71 

4.  I  believe  the  longer  an  IT  manager  is  in  a 
position,  the  more  it  will  affect  then- 
decision  to  outsource 

1  2  3  4  5 
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Level  of  Agreement 

4.00 

5.  I  believe  the  more  other  government 

agencies  outsource  their  IT,  the  more  mine 
might 

1  2  3  4  5 

3.71 

6.  I  believe  outsourcing  activities  are  not 
initiated  by  IT  organizations.  External 
actors  loom  large  in  decisions 

1  2  3  4  5 

3.86 

7.  I  believe  some  IT  managers  view 

outsourcing  as  a  means  to  adopt  industry 
best  practices  and  improve  processes 

1  2  3  4  5 

4.00 

8.  I  believe  the  cost  of  outsourcing  is  not 
clearly  understood 

1  2  3  4  5 

4.43 

9.  I  believe  outsourcing  should  not  be  done 
without  a  comprehensive  business 
assessment  to  establish  value 

1  2  3  4  5 

3.71 

10. 1  believe  managers  do  not  determine  best 
value  prior  to  outsourcing 

1  2  3  4  5 

Thank  you  for  your  time. . . 
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Appendix  D:  Raw  data  from  round  #1 

This  appendix  contains  the  raw  data  form  the  first  round  of  the  Delphi  study. 
Please  note  that  while  the  numbering  scheme  presented  corresponds  to  the  numbering 
used  in  the  instruments  themselves,  they  do  not  match  to  the  scheme  presented  in  the  text. 
To  match  the  two  schemes,  please  refer  to  Appendix  G  for  the  inter-round  question 
numbering  map. 
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Question  1  Item  I  Expert  I  Expert  I  Expert  I  Expert  1  Expert  Expert  Expert  Expert  Expert 


Question  Item  Expert  Expert  Expert  Expert  Expert  Expert  Expert  Expert  Expert  Expert 

a  Tvne  #1  #2  #3  #4  #5  #6  #7  #8  #9  #10 


Appendix  E:  Raw  data  from  round  #2 
This  appendix  contains  the  raw  data  form  the  second  round  of  the  Delphi  study. 
Please  note  that  while  the  numbering  scheme  presented  corresponds  to  the  numbering 
used  in  the  instruments  themselves,  they  do  not  match  to  the  scheme  presented  in  the  text. 
To  match  the  two  sehemes,  please  refer  to  Appendix  G  for  the  inter-round  question 
numbering  map. 
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Expert  #2 


Question  # 
T2-15 


Appendix  F:  Raw  data  from  round  #3 
This  appendix  contains  the  raw  data  form  the  third  round  of  the  Delphi  study. 
Please  note  that  while  the  numbering  scheme  presented  corresponds  to  the  numbering 
used  in  the  instruments  themselves,  they  do  not  match  to  the  scheme  presented  in  the  text. 
To  match  the  two  schemes,  please  refer  to  Appendix  G  for  the  inter-roxmd  question 
numbering  map. 
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Appendix  G:  Inter-round  question  numbering  map 

This  appendix  contains  the  map  to  match  the  question  schemes  used  in  each  of  the 
three  surveys  with  the  universal  coding  used  in  the  text. 
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Appendix  H:  List  of  all  factors,  sub-factors,  and  associated  alphanumeric 

designators 

This  appendix  contains  a  listing  of  all  the  factors,  sub-factors  and  assigned 
alphanumeric  designators. 
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FINANCIAL  (F1-F4) 

BENEFITS:  FINANCIAL 

1 .  Cost  Reductions  =F  1 

a.  Economies  of  scale 

a.  Tighter  control  over  fringe  benefits 

b.  Relocation  of  data  centers  to  lower  cost  areas 

c.  More  focused  ejqjertise  in  managing  IT. 

2.  Improved  cost  controls:  Develop  fi)rmal  and  structured  IT  service  channels  F2 

3.  Restructuring  IT  Budgets:  Using  different  colors  of  money  =  F3 

4.  Protect  and  stabilize  IT  Budgets:  Contracting  regulations  provide  protection  =  F4 
RISKS:  FINANCIAL 

5.  Potential  problems  with  cost  reductions 

a.  Increased  organizational  costs 

b.  Increased  Coordination  costs 

c.  Increased  Transactional  costs. 

d.  Over  inflated  and  inaccurate  cost  savings  projections 

e.  Hidden  costs 

f  Decreased  service 

BUSINESS  (B1-B7) 

BENEFITS:  BUSINESS 

1 .  Focus  on  Core  Competency:  Those  things  a  company  does  better  than  any  other  = 
B1 

a.  Potentially  leads  to: 

L  Simplifying  the  management  agenda 

ii.  Flattening  organizations 

iii.  Leveraging  innovation 

iv.  Increased  employee  motivation  and  cohesion 

V.  Freeing  up  resources  m  the  form  of  funding  and  personnel  to 
apply  to  more  strategic  activities 

vi.  Increased  hardware  utilization 

vii.  Improved  manufacturing  yield 

viii.  Productivity  improvements 

2.  Government  Regulations  and  guidelines  =  B2 

a.  Requirements  to  outsource 

b.  Using  outsourcing  as  a  method  to  demonstrate  compliance  with  guidelines 

3.  Facilitating  Reorganizations  =  B3 

a.  Change  Agent 

b.  Unite  dissimilar  systems 

i.  Solve  technical  incompatibilities 

ii.  Absorb  excess  IT  assets 

iii.  Absorb  unnecessary  IT  employees  generated  by  a  merger 
RISKS:  BUSINESS 

4.  Inability  to  write  an  adequate  service  level  agreement  =  B4 

a.  Inability  to  establish  information  requirements 
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b.  Inability  to  transform  requirements  into  deliverables 

5.  Inability  to  develop  proper  acquisition  strategy 

6.  Inability  to  manage  a  contract:  Using  unskilled  contract  managers  =  B5 

7.  Inability  to  My  leverage  IT:  Less  potential  to  realize  and  understand  the  true 
power  of  IT  =  B6 

a.  Inability  to  get  IT  to  act  as  a  ‘profit  center’ 

8.  Loss  of  control  =  B7 

a.  Lose  control  of  services  provided 

b.  Critical  IT  skills  disappear  fi*om  firm 

c.  Becoming  overly  dependent  on  the  contractor  and  losing  control  of  the 
firm 

d.  Timing  and  quality  of  outputs 

TECHNICAL  (T1-T3) 

BENEFITS:  TECHNICAL 

1.  Improving  Technical  Services:  Dissatisfaction  with  current,  in-house  IT  support  = 
T1 

2.  Access  to  Technical  Talent  =  T2 

a.  Ability  to  find  competent  IT  professionals 

b.  Ability  to  retain  competent  IT  professionals 

c.  Ability  to  reassign  current  internal  IT  staff  members  to  higher  priority 
activities 

3 .  Access  to  New  T echnologies  =  T3 

a.  Access  to  a  vendor’s  large  research  and  development  efforts. 

RISKS:  TECHNICAL 

4.  Potential  problems  with  accessing  technical  talent 

a.  Same  personnel  used 

b.  Talent  is  not  adept  at  solving  a  new  customers  requirements 

c.  Vendor  does  not  upgrade  or  keep  staff  current 

5.  Potential  Problems  with  accessing  new  technologies 

a.  Vendor  and  customer  interfaces  get  complicated  as  systems  grow 

6.  Inadequate  recovery  and  back-up  capability 

7.  Excess  Integration  and  Interoperability  problems 

POLITICAL  (P1-P6) 

BENEHTS:  POLITICAL 

1.  Proving  efficiency:  Obtaining  outsourcing  proposals  in  the  hopes  that  it  will  prove 

the  efficiency  of  the  internal  IT  departments  and  justify  its  continued  existence  = 

PI 

2.  Justifying  New  Resources:  Getting  senior  leaders  to  authorize  additional  funding 
for  an  IT  program  by  showing  them  outsourcing  is  not  more  cost  effective  =  P2 

3.  Exposing  Exaggerated  Claims:  Work  to  ensure  senior  leaders  do  not  enter  an 
outsourcing  relationship  prematurely  =  P3 

4.  Eliminating  a  Troublesome  Function:  IT  is  not  worth  the  headaches  —  P4 
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5.  Breaking  the  so-called  Glass  Ceiling:  Attempt  by  IT  managers  to  appear  more 
corporate  and  creditable  in  the  hopes  of  enhancing  their  own  career  =  P5 

RISKS:  POLITICAL 

6.  Lack  of  Trust  between  Government  and  Contractors  =  P6 

SECURITY  (S1-S3) 

RISKS:  SECUIRTY 

1 .  Controlling  access  to  the  information  =  S 1 

a.  Granting  vendors  access  to  the  IT  infrastructure 

i.  Using  outside  security  specialists 
1.  Disclosing  classified  data 

b.  Security  concerns  surrounding  sub-contractors 

2.  Overall  vendor  security  procedures  =  S2 

a.  How  they: 

i.  Handle  customer  data  on  their  own  systems 

ii.  Correspond  with  each  other  via  e-mail 

iii.  Dispose  of  a  customers  information  after  it  is  no  longer  needed, 

3.  Release  of  Competitive  advantage  =  S3 

a.  Release  of  system  design  information 

ENVIRONMETAL  (E1-E3) 

BENEFITS:  ENVIRONMETAL 

1 .  Length  of  service  remaining  for  outsource  decision  makers:  The  length  of  time  an 
IT  manager  knows  they  will  be  in  a  position  will  effect  their  decision  to  outsource 

=  E1 

2.  Changes  in  the  amount  of  IT  outsourcing  within  the  DoD:  Diffiision  theory  -  E3 

a.  When  a  respected  firm  within  a  social  system  outsources  its  IT,  other  firms 
follow 

RISKS:  ENVIRONMETAL 

3.  Internal  Resistance:  Concern  over  displaced  IT  staff  members  effects  a  firms 
decision  to  outsource  =  E2 


NF,W  FACTORS  CREATED  BY  EXPERTS 

NEW  ENVIRONMETAL 
BENEHTS:  ENVIRONMETAL 

1 .  Intra  Agency  Relationships 

2.  External  Actors 

NEW  ADDITIONAL 

BENEFITS:  ADDITIONAL 
1 .  Best  practices  and  processes 
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